## Optimal and Suboptimal Signal Detection-On the Relationship Between Estimation and Detection Theory

##### Abstract

In this thesis, we consider a general binary and M-ary hypothesis testing problems with unknown parameters. In a hypothesis testing problem, we assume that the probability density function (pdf) of observation is given while some unknown parameters exit in the structure of the pdf and the set of unknowns under each hypothesis is given.
In the first part of this study, for a binary composite testing, we prove that the Minimum Variance and Unbiased Estimator of a Separating Function (SF) serves as the optimal decision statistic for the Uniformly Most Powerful (UMP) unbiased test. In many problems, the UMP test does not exist. For such cases, we introduce new suboptimal SF-Estimator Tests (SFETs) which are easy to derive for many problems.
In the second part, we study the relationship between Constant False Alarm Rate (CFAR) and invariant tests. We generally show that for a family of distributions, the unknown parameters are eliminated from the distribution of the maximal invariant statistic under the Minimal Invariant Group (MIG) while the maximum information of the observed signal is preserved. We prove that any invariant test with respect to the MIG is CFAR. Then, we introduce the UMP-CFAR test as the optimal CFAR bound among all CFAR tests.
In the third part, the asymptotical optimality of the CFAR tests is studied after reduction using MIG. We show that the CFAR tests obtained after MIG reduction using the Wald test is SFET and the Generalized Likelihood Ratio Test and the Rao test are asymptotically optimal. To find an improved test, we maximize the asymptotic probability of detection of the SFET using the Maximum Likelihood Estimation (MLE). We propose a systematic method allowing to derive the Asymptotically Optimal SFET. Finally, we extend the concept of SFET to the M-ary hypothesis testing. Defining M different SFs for an M-ary problem, we show that the optimal minimum error test is achieved using the MLE of SFs. Moreover, in the case that the optimal minimum error test does not exist, the error probability of the proposed SFET tends to zero when the number of independent observations tends to infinity.