Multidimensional Signal Space Partitioning Using a Minimal Set of Hyperplanes for Detecting ISI-Corrupted Symbols

Transcription

1 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL 48, NO 4, APRIL Multidimensional Signal Space Partitioning Using a Minimal Set of Hyperplanes for Detecting ISI-Corrupted Symbols Younggyun Kim and Jaekyun Moon Abstract A signal space partitioning technique is presented for detecting symbols transmitted through intersymbol interference channels The decision boundary is piecewise linear and is made up of several hyperplanes The goal here is to minimize the number of hyperplanes for a given performance measure, namely, the minimum distance between any signal and the decision boundary Unlike in Voronoi partitioning, individual hyperplanes are chosen to separate signal clusters rather than signal pairs The convex regions associated with individual signals, which together form the overall decision region, generally overlap or coincide among in-class signals The technique leads to an asymptotically optimum detector when the target distance is set at half the minimum distance associated with the maximum-likelihood sequence detector Complexity and performance can be easily traded as the target distance is a flexible design parameter Index Terms Fixed-delay tree search, intersymbol interference, MLSD, sequence detection, signal space partition I INTRODUCTION THIS PAPER is about a sequence detection technique for channels whose major impediments are intersymbol interference (ISI) and additive Gaussian noise Many communication channels suffer from ISI due to bandwidth limitation While the detrimental effect of ISI on the performance of symbol detection can be eliminated or reduced by using the maximum-likelihood sequence detector (MLSD) [1], the required implementation complexity may be very high Also, the performance of the MLSD is guaranteed in general only with a sufficient decision delay [2] There exists a number of techniques which attempt to reduce the complexity of MLSD at the expense of some loss of optimality Some of these techniques are based on effectively reducing the size of the trellis with the use of decision feedback [3] [5], but without constraining decision delay Some others impose a fixed decision delay to limit the number of observation samples used in the detection process, thereby reducing and fixing the processing requirements per symbol period [6] [10] Paper approved by R A Kennedy, the Editor for Data Communications Modulation and Signal Design of the IEEE Communications Society Manuscript received May 3, 1998; revised January 4, 1999 and September 14, 1999 This work was supported by the National Science Foundation under Grant CCR , Texas Instruments, Hitachi, and Marvell This paper was presented in part at the ICC 98, Atlanta, GA, June 1998, and in part at the 1998 International Conference on Magnetics, San Francisco, CA, January 1998 Y Kim is with the Storage Products Group, Texas Instruments, Tustin, CA USA J Moon is with the Department of Electrical and Computer Engineering, University of Minnesota, Minneapolis, MN USA ( moon@eceumnedu) Publisher Item Identifier S (00) In this paper, we focus our attention on detectors subject to a fixed decision delay and the formulation of the detection process in signal space, which often leads to efficient implementation For certain channels, sequence detectors with a finite, relatively small decision delay can achieve MLSD or near-mlsd performance [11], [8], [12], [10] For other channels, the finite-delay detectors provide tradeoff options between complexity and performance Depending on the chosen delay, the detection quality changes from that of the decision-feedback equalizer to that of the MLSD There has been considerable interest in implementing finite-delay detectors using signal space partitioning [13] [18] Signal space interpretations of decision boundaries also have been given for constrained-delay optimal detectors utilizing past decisions [9], [19] For optimal performance, the decision boundary is in general nonlinear [9] Among the previously studied techniques, the work of [18] describes a signal space partitioning method based on linear decision boundaries which is mathematically equivalent to the fixed-delay tree search detector of [8] Under the assumption of correct past decisions, this detector provides asymptotically optimal decision quality for the given delay constraint (the delay-constrained optimal decision could be obtained by the detector of [20] with or without correct past decisions, but at the cost of significant increase in complexity) The method of [21] can also be used to find a set of hyperplanes for detecting symbols in the presence of ISI This paper presents a different partitioning method which provides improved performance for a given complexity level as well as more flexible performance/complexity tradeoffs In [18], which was inspired by an earlier work of [22], all possible finite-length signal sequences are first mapped to a multidimensional vector space Hyperplanes which form nonoverlapping Voronoi regions are then obtained, and the overall decision region for a given decision class is given by the union of all in-class Voronoi regions A Voronoi region is formed by separating a given signal with its immediate neighbors (known as Delaunay neighbors) using orthogonally bisecting hyperplanes In the present paper, we form the final decision boundary using hyperplanes that separate clusters of signals rather than pairs of signals As such, convex regions associated with individual in-class signals often coincide with one another, and many different pairs of opposite-class signals can share a common separating hyperplane This results in a reduced number of hyperplanes Finding a single hyperplane that max /00$ IEEE

2 638 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL 48, NO 4, APRIL 2000 Fig 1 Structure of signal space detector imally separates two scattered sets of opposite-class signals has been considered in [17] This paper addresses a multidimensional, many-hyperplane problem Like any signal space detector with a piecewise-linear decision boundary, the detector structure consists of parallel linear discriminant functions and a many-to-one Boolean mapper Thus, the detector design amounts to finding a minimal set of discriminant functions and appropriate Boolean mapper for a prescribed minimum distance At high signal-to-noise ratios (SNR s), the specified minimum distance determines the performance of this detector The prescribed distance also controls the number of required hyperplanes In Section II, we review the signal space detector and its structure Section III describes the proposed signal space partitioning technique that leads to a minimal set of hyperplanes for the given target distance As examples, some channels with a finite impulse response are examined in Section IV The detector performance is analyzed in Section V based on the upper and lower bounds of the symbol-error probability II SIGNAL SPACE DETECTION The discrete-time channel model we consider here is represented by where is an observation sample, represents the overall channel response is the input symbol taken from, and is zero-mean additive white Gaussian noise We limit our presentation to binary signaling for the sake of clarity, but there exist no conceptual difficulties in extending the proposed technique to multilevel inputs A signal space detector with a decision delay of makes a decision on symbol at time based on observation samples Past decisions on the input symbols, are used to cancel ISI terms from observation samples In this process, past decisions are assumed to be correct After canceling ISI (1) from past input symbols, the observation samples available at the detector input are given by (2) where is the noiseless signal The detector finds a noiseless signal vector which maximizes the probability for a given observation sample vector and releases the associated as the symbol decision Since noise samples are assumed white Gaussian, the selected signal vector is the one nearest to in the Euclidean sense This decision process can be viewed as partitioning the -dimensional observation space into appropriate nonoverlapping decision regions The corresponding decision boundary is piecewise linear, which can be represented by a set of hyperplanes The resulting structure for the signal space detector is shown in Fig 1 A finite number of channel output samples are the input to the linear discriminant functions, where the ISI components contributed by previously detected symbols are removed in advance from each sample Each linear discriminant function represents a hyperplane in a -dimensional signal space A threshold detector determines which side of the corresponding hyperplane the observation vector is located Finally, a Boolean logic function estimates the channel input symbol based on the location of the observation vector relative to each hyperplane In this structure, the complexity of a detector is mainly determined by the number of hyperplanes Since the asymptotic bit-error rate performance of a signal space detector with a finite-decision delay is determined by the minimum distance from any signal vector to the decision boundary, this distance will be used as a performance measure to obtain a reduced complexity detector That is, a given minimum distance is preserved while obtaining a minimal set of hyperplanes which form the overall decision boundary III SIGNAL SPACE PARTITIONING Let and be sets of noiseless signal vectors corresponding to symbol decision (class-1) and (class-2), re-

3 KIM AND MOON: MULTIDIMENSIONAL SIGNAL SPACE PARTITIONING USING A MINIMAL SET OF HYPERPLANES 639 spectively The procedure for obtaining the minimum number of hyperplanes for signal space detection is described as follows 1) For each, all possible subsets are obtained, excluding the null subset 2) All possible pairs of subsets of opposite decision classes are formed A subset-pair consists of two subsets and 3) For each subset-pairing, a hyperplane which separates two subsets and is obtained so as to maximize the minimum distance from any signal in to the hyperplane Only those hyperplanes which yield the minimum distance greater than or equal to a prescribed value are retained for the next step The specified distance determines the performance of the detector 4) From the chosen hyperplanes, a minimum number of hyperplanes are obtained by which every pair of oppositeclass signals can be separated with the prescribed distance 5) A Boolean logic function is obtained to make a final decision based on the location of the observation vector relative to each hyperplane in the minimal set This is done by first defining a convex region associated with each signal for a given class and then forming a union of these regions Unlike in direct Voronoi partitioning, these convex regions frequently coincide with one another When the procedure is done, every pair of opposite-class signals will be separated by distance no less than by at least one hyperplane in the minimal set The procedure eliminates the case where a pair of opposite-class signals are separated by two or more hyperplanes Therefore, the resulting hyperplanes form a minimal set under the constraint that no more than one hyperplane is used to separate a given pair of opposite-class signals and a hyperplane can be shared among different pairs of opposite-class signals In Step 2), even for a small number of signal vectors, the number of subset-pairs will be quite large In Section III-B, a technique is outlined which reduces the number of subset-pairs that need be examined Note that since the last element of each signal vector is either or according to the input symbol, the signal vectors in each subset-pair are always linearly separable into two decision classes In Step 3), a gradient descent procedure given in Section III-A is used to obtain a hyperplane for each subset-pair In Section III-C, we show that Step 4) is essentially the well-known set covering problem Section III-D presents a method to obtain a Boolean logic function for Step 5) For a large decision delay (ie, a large number of signals), the proposed procedure may not be practical for direct application For such cases, we propose an incremental partitioning method to control the computational intensity This is given in Section III-E A Gradient Descent Search for a Hyperplane For the time being, let us focus on finding a single hyperplane which will separate two linearly separable signal sets Since the minimum distance from any noiseless signal to the decision boundary determines the performance of the detector, a hyperplane is obtained so as to maximize this minimum distance In a vector space in which resides, a hyperplane is represented by where and The vector and are the coefficients of a hyperplane The problem we are dealing with is a classification of two classes As shown in Fig 1, a threshold detector is used to classify a signal That is, for a class-1 signal is greater than or equal to zero, and for a class-2 signal is less than zero The problem of finding a hyperplane can be stated as where and is an indicator function such that for a class-1 signal and for a class-2 signal If we define as where can be written as (3) (4) (5), the criterion function to be maximized Accordingly, the gradient descent procedure can be written as (7) where is the step size and Strictly speaking, the derivative of the function does not exist at those values of where more than one signal have the same cost function For these values of in (7) is replaced by for any one of those signals To make the discussion of the convergence properties simple, the constant term can be removed from the hyperplane with no loss of generality This can be achieved by shifting all signals appropriately so that With the constraint of and, the function reduces to (8) where the coefficients is located on the unit hypersphere represented by in the coefficient space For a given signal, the function has unique maximum and minimum under the constraint, which are and, respectively Let us consider only a positive value of the function For each signal, the function with a positive-value constraint is convex for on the unit hypersphere It is not difficult to see that the minimum of these convex functions is also convex So the function has the unique maximum and is convex on the unit hypersphere To enforce the constraint in (7) need be normalized by By choosing an appropriate initial value for, the positive value constraint can easily be accommodated Since the classification problem we consider here is linearly separable, the initial value of can be chosen such that for every is positive Also note that for the maximization problem at hand, does not become zero at the global maximum (6)

4 640 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL 48, NO 4, APRIL 2000 This, however, does not pose a problem in terminating the gradient search of (7) as the search can end when ceases to change by a significant amount We finally note that a linear programming technique is also possible for separating signal clusters [17] B Upper Bound on the Number of Subset-Pairings If the size of a set is denoted by, the total number of the subset of a set is The net number of the subset-pairs corresponding to opposite decisions is The following lemma is used to establish an upper bound on the number of the subset-pairs to be examined Let us first define the convex hull A set in a -dimensional space is convex if for any pair of points, the line segment with end points and is contained within The convex hull of a set, is the smallest convex set that contains the set Lemma: In a -dimensional vector space, no more than signals are necessary to define a hyperplane which separates a pair of linearly separable signal sets with the criterion of maximizing the minimum distance Proof: We can represent (4) in a simple form by introducing a -dimensional vector which is obtained from a -dimensional vector via the mapping [23] for class-1 signal for class-2 signal (10) Using this mapping, (4) reduces to subject to (11) The hyperplane defined in the -dimensional vector space passes through the origin of the coordinates and has all the signals in one side of it Without losing generality, we can assume that the coefficient, which can be achieved by shifting all signals and the hyperplane appropriately in the -dimensional space Let us denote the coefficients of a hyperplane obtained from (11) by and a set of signals which are located at minimum distance from the hyperplane by, where is the number of these signals A point defined by for any is the closest point in to the origin Note that Once a hyperplane is defined by (11), a signal is redundant in specifying the hyperplane This is clear since the signals that are further away will not affect the hyperplane To further remove redundant signals in, let us first assume that, spans a -dimensional space For a -dimensional space in which resides, the smallest subset whose convex hull includes the point has at most signals This is still the closest point from the origin to the convex hull So signals are sufficient to define Since the maximum dimensionality which can be spanned by (9), is, at most signals are sufficient to define From the above lemma, a proposition follows which gives the upper bound on the number of the subset-pairs need be examined Proposition: For a -dimensional vector space, the total number of subset-pairs to be searched is at most (12) Accordingly, only those subset-pairs in which the total number of signals is in the range of 2 through are examined For the most complex example examined here, the number of the subset-pairs to be examined is reduced by more than a factor of 2 C Obtaining a Minimal Set of Hyperplanes Each hyperplane and pair of opposite-class signals are represented by, and, respectively is the total number of the hyperplanes and The problem of finding the minimum number of hyperplanes which can separate every signal-pairs can be represented in the following matrix form: (13) where Each column and row correspond to a hyperplane and a signal-pair, respectively If a hyperplane can separate a signal-pair, the element is 1 Otherwise, is 0 The problem in this step reduces to finding the minimum set of columns which have at least one 1 in each row This can be viewed as a set covering problem in graph theory and can be solved by a tree search algorithm [24] If and, the problem finds a set of hyperplanes such that (14) and is minimum This problem can also be solved using an integer programming method [25] when formulated in the following form: subject to (15) where is either 1 or 0 according to or, respectively The number of hyperplanes and signal-pairs can be further reduced by some preprocessing prior to applying the tree search algorithm If and, the hyperplane can be removed from the consideration because every signal-pair in is separated by the hyperplane Similarly, if

5 KIM AND MOON: MULTIDIMENSIONAL SIGNAL SPACE PARTITIONING USING A MINIMAL SET OF HYPERPLANES 641 and, the signal-pair can be removed because any hyperplane which separates separates If the prescribed distance is less than or equal to half the minimum distance between signals in any, the solution of this problem is guaranteed because a hyperplane defined by each is included in the initial set of hyperplanes D Boolean Logic Function At this stage, we have a minimal set of hyperplanes by which every opposite-class signal-pair can be separated with the prescribed distance The half-space that is defined by and associated with a class-1 signal is represented by corre- Since every signal-pair can be separated, the region sponding to a class-1 signal is obtained by (16) (17) where The region for class-1 signals is the union of for all Note that unlike in the Voronoi partition [18], 's may coincide If a threshold detector output corresponding to a linear discriminant function has Boolean logic value 1 and 0 according to and, respectively, a Boolean logic value indicating whether or not can be obtained via a logical ANDoperation of Similarly, a Boolean logic value indicating whether is located in the region corresponding to class-1 signals or not can be obtained via a logical ORoperation of for all E Incremental Method The number of signals in increases exponentially as the decision delay increases (or as the dimension of the signal space increases) For higher dimensional space, the total number of subset-pairs may be so large that direct application of the proposed procedure in the previous section is impractical due to its computational complexity and memory requirement Since the detector performance depends on the decision delay, a certain value, say, is required to achieve the desired detector performance [12] An incremental method to handle a large case can be described as follows 1) A small value is chosen for which the proposed procedure can be applied directly 2) For the chosen : a) if, some pairs of opposite-class signals may not be separated with the prescribed distance The proposed procedure is applied to obtain a minimal set of hyperplanes which separate the remaining pairs of signals; b) for each pair of signals not separable, is increased by and Step 2) is repeated For random input data, and are both From (12), the total number of subset-pairs to be searched is given by (18) For and is 6448 and , respectively The initial in Step 1) is chosen so that is small enough for the proposed procedure to be reasonable in terms of complexity The total number of hyperplanes to be obtained is also Additionally, the number of signal-pairs and hyperplanes in the set covering problem have to be considered for choosing a manageable In Step 2a), for, signals in some opposite-class signal pairs are separated by less than, where is the required minimum distance from signals to the decision boundary This is clear because if signals in any pair are separated by distance no less than can be reduced to without degrading performance in the minimum distance sense Let be the set where the signals and are separated by less than By applying the proposed procedure, a minimal set of hyperplanes which separate the remaining pairs of opposite-class signals,, can be obtained The pair would be separated for larger value (or in higher dimensional space) Let the Boolean logic value which will be defined for a larger value indicate that the observation sequence is in the region for rather than in the region for, where The Boolean logic value indicating whether or not is obtained by a logical ANDoperation of and for all As in the previous section, the final decision is obtained by a logical ORoperation of for all By increasing by, each signal for a delay generates descendants (assuming random input data), where a signal vector for a delay forms the lower elements of every signal vector in Therefore, a pair of signals generates pairs of subsets can be used to choose a manageable value in Step 2b) Note that the incremental method outlined above generally do not result in a minimal set of hyperplanes IV EXAMPLES In this section, the procedure is applied to a partial response (PR) channel driven by the minimum run length codes [26] and two other ISI channels of lengths 3 and 4 driven by random input With the constraint, the input sequence cannot have consecutive symbol changes; no input sequence contains or patterns This type of code finds wide application in both magnetic and optical recording channels [27], [28] A PR2 Channel The PR2 channel is specified by the response The MLSD for the PR2 channel with the constraint gives rise to the minimum distance (half the minimum

6 642 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL 48, NO 4, APRIL 2000 Fig 2 Signal space detector for PR2 channel TABLE I HYPERPLANES AND NOISELESS SIGNALS FOR PR2 CHANNEL distance between noiseless signal sequences of any length) of A signal space detector with yields the same minimum distance as the MLSD For this example, is chosen to achieve the same performance as the MLSD For the previous decision, the number of noiseless signals associated with and are 5 and 3, respectively, after eliminating any symbol sequences that contain or pattern The number of subset-pairs to be examined is 180 Among 180 corresponding maximally separating hyperplanes, only those hyperplanes for which the minimum distance is greater than or equal to are kept for further processing The minimum number of hyperplanes which can separate all signal-pairs turned out to be 2 Table I shows the hyperplanes and their separable noiseless signals A 1 means that the signal is separable by the corresponding hyperplane with minimum distance no less than A 0 means the noiseless signal is not separable by the hyperplane With the convention used in Table I, to have a signal-pair separated by a hyperplane, both signals in the pair must be separable by the hyperplane Note that the notation here is made somewhat different from (13) for the sake of brevity The hyperplane separates the signal from signals and and the hyperplane separates the signal from Therefore, the convex region for is obtained by the intersection of half-spaces and defined by and, respectively Similarly, the convex regions for can be obtained and are shown to coincide with Accordingly, the union of these regions is also Hence, the Boolean logic function is a logical ANDof two-threshold detector outputs The final detector structure is shown in Fig 2 The signchange by a multiplication with and exclusive ORgate are needed to handle the case This simple incorporation is possible due to the symmetry between the noiseless signals for and Application to a magnetic recording channel has also been considered in a separate paper [29] B Channel A Channel A is characterized by the response Random input data is assumed is used for this example For the chosen decision delay, any input error sequences of the form result in the minimum distance For this channel, the maximally achievable minimum distance without a delay constraint is which is generated by the error sequences of the form The optimality loss of the signal space detector is therefore 013 db This is a compromise we opted for in this example For, the numbers of noiseless signals corresponding to and are both 4 The number of subset-pairs to be examined is 132 By applying the proposed procedure directly, five hyperplanes are obtained which can separate every pair of opposite-class signals with a distance no less than These hyperplanes and noiseless signals are shown in Table II The hyperplane separates the signals and from any signals in class-2 The regions and for the signals and, respectively, coincide with the half-space defined by the hyperplane The signal requires three hyperplanes, and to be separated from the signals in class-2 The region for is the intersection of the half-spaces, and defined by the hyperplanes, and, respectively For the signal, the hyperplanes and are enough to separate from any signals in class-2 Therefore, the region for is the intersection of the half-spaces and defined by and, respectively The region for the decision is the union of these regions, defined for the class-1 signals The resulting signal space detector for this channel is shown in Fig 3 C Channel B Channel B has the response Again, a random input symbol sequence is assumed ISI in this channel is relatively severe, and in the

7 KIM AND MOON: MULTIDIMENSIONAL SIGNAL SPACE PARTITIONING USING A MINIMAL SET OF HYPERPLANES 643 Fig 3 Signal space detector for channel A TABLE II HYPERPLANES AND NOISELESS SIGNALS FOR CHANNEL A TABLE III HYPERPLANES AND NOISELESS SIGNALS FOR CHANNEL B absence of any code constraint which simplifies the signal constellation, the complexity of resulting signal space detector is significantly larger than that of the two previous examples Nevertheless, this channel is chosen to demonstrate the incremental partitioning technique A reasonable choice on the delay for this channel appears to be ; for, the error sequences result in the minimum distance which represents a 055-dB performance loss relative to the minimum distance of the MLSD The required to achieve the same performance as the MLSD is 8, for which the error sequences are This would require exceedingly high complexity and the use of the Viterbi algorithm (VA) would be more sensible over any restricted delay detectors, if the goal is to achieve the MLSD performance [2] With, the number of signals associated with either or is 16, and the number of subset-pairs to be examined is ! Instead of direct application of the proposed technique to is considered first In the lower dimension corresponding to, the number of signals is 4 for each class The corresponding number of subset-pairs to be examined is 132 Table III shows the separating hyperplanes and the signals that are separable by these hyperplanes In the signal-space of, three signal-pairs,, and, are not separable with distance no less than These signal-pairs need to be considered in a higher dimension The signal-pair requires and the pairs and require for separation with the prescribed distance of TABLE IV HYPERPLANES AND NOISELESS SIGNALS FOR SIGNAL-PAIR (s ; s ) OF CHANNEL B In the signal-space of, the signal-pair gives rise to four signals The corresponding hyperplanes and the separable descendants are shown in Table IV The region for is the half-space defined by and the region for is the intersection of the half-spaces and defined by and, respectively The union of these two regions is the convex region associated with the signal in the current signal-space of This region for defines a Boolean logic value which is used to separate from the four opposite-class signals in the space The signal-pair results in eight descendants in the signal-space of Five hyperplanes are obtained for these signals and they are shown in Table V The regions for the signals and coincide with the half-space defined by the hyperplane The convex region for is the intersection of and The region for is the intersection of and The union of these regions form the region for in the pair This region for defines a Boolean

8 644 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL 48, NO 4, APRIL 2000 TABLE V HYPERPLANES AND NOISELESS SIGNALS FOR SIGNAL-PAIR (s ; s ) OF CHANNEL B TABLE VI HYPERPLANES AND NOISELESS SIGNALS FOR SIGNAL-PAIR (s ; s ) OF CHANNEL B V PERFORMANCE ANALYSIS The performance of a fixed-delay sequence detector is a strong function of the decision delay Since the PR2 channel has a spectral null at Nyquist frequency, the so-called quasi-catastrophic error events can occur With the code constraint, the quasi-catastrophic error events and other multiple-symbol error events in the PR2 channel are prevented Therefore, the signal space detector with a finite decision delay can achieve the matched filter bound A decision delay of has been chosen for the PR2 channel to achieve the matched filter bound For channel A, the minimum distance of the MLSD is and this can be achieved by a signal space detector with For channel B, the MLSD distance is achieved with a finite decision delay of As a compromise between complexity and performance, we opted for decision delays of and for channels A and B, respectively The associated performance losses relative to MLSD are 013 and 055 db, respectively The upper bound for the symbol-error probability for the proposed signal space detector can be easily obtained from its signal space representation In the following derivation, the noise is additive white Gaussian with zero mean and variance The conditional pdf of the observation vector given is given by (19) logic value which is used to separate from any signals in class-2 in the signal-space corresponding to Similarly, the signal-pair results in eight signals in the signal-space of and the number of hyperplanes obtained is five Table VI shows the hyperplanes and their separable signals The regions for and are the same as The region for is the intersection of and The region for is the intersection of and The union of these regions forms the region for in the pair This region for defines a Boolean logic value which is used to separate from any signals of class-2 in the signal space In Table III, the Boolean logic values for and are the same as, the output of the threshold detector corresponding to The Boolean logic value for is a logical ANDof, and Similarly, the Boolean logic value for is a logical ANDof, and A logical ORof these outputs results in the logic value corresponding to the decision The resulting detector is shown in Fig 4 Note that the detector structures shown in Figs 2 4 can be modified slightly so that the feedback cancellation of past decisions can be moved toward the end of the processing stage This will result in a structure that is well suited to high speed pipelining It is also evident from the figures that the detection process is highly parallel The Boolean mapper consisting of ANDand ORoperations can also be implemented using a single lookup table where is a Gaussian pdf with mean and diagonal covariance matrix with The conditional pdf of given can be defined similarly When error propagation is neglected, the error probability is given by (20) where is the region corresponding to the symbol decision and is the union of for all The upper bound of can be obtained from the union bound of (20), shown in (21), at the bottom of the next page, where is the region for a class-2 signal is the number of hyperplanes which define the region or, corresponding to is the distance from the signal to the hyperplane is the minimum distance from any signal to the decision boundary, is the total number of hyperplanes which are associated with the minimum distance with respect to the signal, and The first inequality in (21) is achieved due to the enlarged integration region for each signal and the second inequality is the union bound The approximation in (21) is valid for high SNR s Note that unlike in [1], [30], and [12], the distance is defined as the minimum distance from any signal to the decision boundary rather than from signal to signal Also note that in (21) is the number of hyperplanes rather than the number of minimum distance neighboring signals as in [1] For a given decision delay, the prescribed distance used in the proposed procedure

9 KIM AND MOON: MULTIDIMENSIONAL SIGNAL SPACE PARTITIONING USING A MINIMAL SET OF HYPERPLANES 645 Fig 4 Signal-space detector for channel B cannot be larger than half the minimum distance between any opposite-class signals If for a given, the genie-guided detector described in [12] is optimal Therefore, the lower bound of [12] can serve as the lower bound of the present signal space detector (22) where is a set of signals which have at least one opposite-class signal at distance The upper and lower bounds for the examples are shown in Fig 5 For the upper bound calculation, the union bound is used in (21) (the second inequality) instead of the high SNR approximation Bit-error rate simulation results are also denoted as circles The SNR for each channel is defined as the ratio of the signal power for single data input and the noise variance From the upper and lower bounds shown, it is clear that if for a given, where is the minimum distance between any noiseless signals [1], [30], the performance of the signal space detector is asymptotically optimal (given the delay constraint) VI CONCLUSION A systematic way of obtaining a reduced complexity signal space detector has been presented The signal space detector (21)

10 646 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL 48, NO 4, APRIL 2000 (a) (b) (c) Fig 5 Upper and lower bounds of symbol error probability (a) PR2 channel (b) Channel A (c) Channel B consists of linear discriminant functions, threshold detectors, and a Boolean logic function The resulting decision boundary is piecewise linear Our goal here was to minimize the number of linear discriminant functions (hyperplanes) for a given performance measure The detector design procedure starts with the opposite-class pairing of all signal subsets Each pair defines a hyperplane separating them By searching through these hyperplanes, a signal space detector with the minimal set of hyperplanes is obtained The procedure has been applied to a coded PR2 channel as well as two other example channels The detector performance has been analyzed by obtaining the upper and lower bounds of the symbol-error probability If the performance measure is set at half the minimum distance between any noiseless signals, the proposed procedure results in a delay-constrained, asymptotically optimal detector A gradient descent procedure also has been presented to obtain a hyperplane so as to maximize the minimum distance between the hyperplane and any signal REFERENCES [1] G D Forney Jr, Maximum-likelihood sequence estimation of digital sequences in the presence of intersymbol interference, IEEE Trans Inform Theory, vol IT-18, pp , May 1972 [2] G D Forney Jr, The Viterbi algorithm, Proc IEEE, vol 61, pp , Mar 1973 [3] J W M Bergmans, S A Rajput, and F A M van de Laar, On the use of decision feedback for simplifying the Viterbi detector, Philips J Res, vol 42, no 4, pp , Jan 1987 [4] A Duel Hallen and C Heegard, Delayed decision-feedback sequence estimation, IEEE Trans Commun, vol 37, pp , May 1989 [5] M V Eyuboglu and S U H Qureshi, Reduced-state sequence estimation for coded modulation on intersymbol interference channels, IEEE J Select Areas Commun, vol 7, pp , Aug 1989 [6] A P Clark, L H Lee, and R S Marshall, Developments of the conventional nonlinear equalizer, Proc Inst Elect Eng, vol 129, no 2, pp 85 94, Apr 1982 [7] J G Proakis and A Khazen-Terezia, A decision-feedback tree-search algorithm for digital communication through channels with intersymbol interference, in Proc Int Conf Commun, June 1986 [8] J Moon and L R Carley, Performance comparison of detection methods in magnetic recording, IEEE Trans Magn, vol 26, pp , Nov 1990 [9] D Williamson, R A Kennedy, and G W Pulford, Block decision feedback equalization, IEEE Trans Commun, vol 40, pp , Feb 1992 [10] J Moon and L R Carley, Efficient sequence detection for intersymbol interference channels with run-length constraints, IEEE Trans Commun, vol 42, pp , Sept 1994 [11] R W Wood, New detector for 1,k codes equalized to class II partial response, IEEE Trans Magn, vol 25, pp , Sept 1989 [12] J W M Bergmans, F M J Willems, and G S M Kerpen, On the performance of data receivers with a restricted detection delay, IEEE Trans Commun, vol 42, pp , June 1994 [13] R W Wood, Enhanced decision feedback equalization, IEEE Trans Magn, vol 26, pp , Sept 1990 [14] A M Patel, A new digital signal processing channel for data storage products, IEEE Trans Magn, vol 27, pp , Nov 1991 [15] J G Kenney, L R Carley, and R W Wood, Multi-level decision feedback equalization for saturation recording, IEEE Trans Magn, vol 29, pp , July 1993 [16] B Brickner and J Moon, A high dimensional signal space implementation of FDTS/DF, IEEE Trans Magn, vol 32, pp , Sept 1996 [17] S A Altekar, A E Vityaev, and J K Wolf, Decision feedback equalization via separating hyperplanes, IEEE Trans Commun, submitted for publication [18] J Moon and T Jeon, Sequence detection for binary ISI channels using signal space partitioning, IEEE Trans Commun, vol 46, pp , July 1998

11 KIM AND MOON: MULTIDIMENSIONAL SIGNAL SPACE PARTITIONING USING A MINIMAL SET OF HYPERPLANES 647 [19] S Chen, B Mulgrew, and S McLaughlin, Adaptive Bayesian equalizer with decision feedback, IEEE Trans Signal Processing, vol 41, pp , Sept 1993 [20] K Abend and B D Fritchman, Statistical detection for communication channels with intersymbol interference, Proc IEEE, vol 58, pp , May 1970 [21] R A Iltis, A randomized bias technique for the importance sampling simulation of Bayesian equalizers, IEEE Trans Commun, vol 43, pp , Feb/Mar/Apr 1995 [22] N K Bose and A K Garga, Neural network design using Voronoi diagrams, IEEE Trans Neural Networks, vol 4, pp , Sept 1993 [23] R Duda and P Hart, Pattern Classification and Scene Analysis New York: Wiley, 1973 [24] N Christofides, Graph Theory: An Algorithmic Approach New York: Academic Press, 1975 [25] H Taha, Operations Research: An Introduction New York: MacMillan, 1976 [26] P H Siegel and J K Wolf, Modulation and coding for information storage, IEEE Commun Mag, vol 29, pp 68 86, Dec 1991 [27] K A S Immink, Coding techniques for the noisy magnetic recording channel, IEEE Trans Commun, vol 37, pp , May 1989 [28] K A S Immink, Coding methods for high-density optical recording, Philips J Res, vol 41, pp , 1986 [29] Y Kim and J Moon, Low complexity signal space detector for (1; 7)-coded partial response channels, IEEE Trans Magn, vol 34, pp , July 1998 [30] G D Forney Jr, Lower bounds on error probability in the presence of large intersymbol interference, IEEE Trans Commun, vol COM-20, pp 76 77, Feb 1972 Younggyun Kim received the BS degree from the Hanyang University, Seoul, Korea, in 1986, and the MS and PhD degrees in electrical engineering from the University of Minnesota, Minneapolis, in 1996 and 1998, respectively From 1986 to 1993, he was with Samsung Electronics, where he worked as a VLSI Design Engineer In 1998, he joined the Storage Products Group of Texas Instruments, Tustin, CA, and is currently working on signal processing for data storage Jaekyun Moon received the BSEE degree from the State University of New York at Stony Brook, in 1984, and the MS and PhD degrees in the Electrical and Computer Engineering Department, Carnegie Mellon University, Pittsburgh, PA, in 1987 and 1990, respectively He held a short-term position at IBM Research in 1987 Since 1990, he has been with the faculty of the Department of Electrical and Computer Engineering at the University of Minnesota, Twin Cities, where he is now a Professor He has also been acting as a consultant for disk drive and tape drive industry since 1990 He received the McKnight Land Grant Professorship from the University of Minnesota His research interests are in the area of channel characterization and signal processing for data storage and digital communication Prof Moon is a member of Tau Beta Pi and Eta Kappa Nu He was selected to receive the IEEE Engineering Foundation Research Initiation Award in 1991, and is a recipient of the IBM Faculty Development Awards as well as the IBM Partnership Awards, and the 1997 National Storage Industry Consortium (NSIC) Technical Achievement Award He served as Program Chair for the 1997 Magnetic Recording Conference He is also Past Chair of the Signal Processing for Storage Technical Committee of the IEEE Communications Society