menu

Sunday, 1 January 2017

The problem with the likelihood ratio for DNA mixture profiles


We have written many times before (see the links below) about use of the Likelihood Ratio (LR) in legal and forensic analysis.

To recap: the LR is a very good and simple method for determining the extent to which some evidence (such as DNA found at the crime scene matching the defendant) supports one hypothesis (such as "defendant is the source of the DNA") over an alternative hypothesis (such as "defendant is not the source of the DNA"). The previous articles discussed the various problems and misinterpretations surrounding the use of the LR. Many of these arise when the hypotheses are not mutually exclusive and exhaustive. This problem is especially pertinent in the case of 'DNA mixture' evidence, i.e. when some DNA sample relevant to a case comes from more than one person. With modern DNA testing techniques it is common to find DNA samples with multiple (but unknown number of) contributors. In such cases there is no obvious 'pair' of hypotheses that are mutually exclusive and exhaustive, since we have individual hypotheses such as:
  • H1: suspect + one unknown
  • H2: suspect + one known other 
  • H3: two unknowns
  • H4: suspect + two unknowns 
  • H5: suspect + one known other + one unknown
  • H6: suspect + two known others
  • H7: three unknowns 
  • H8: one known other + two unknowns
  • H9: two known others + one unknown
  • H10: three known others
  • H11:  suspect + three unknowns 
  • etc.
It is typical in such situations to focus on the 'most likely' number of contributors (say n) and then compare the hypothesis "suspect + (n-1) unknowns" with the hypothesis "n unknowns". For example, if there are likely to be 3 contributors then typically the following hypotheses are compared:
  • H1: suspect + two unknowns
  • H2: three unknowns
Now, to compute the LR we have to compute the likelihood of the particular DNA trace evidence E under each of the hypotheses. Generally both of these are extremely small numbers, i.e. both the probability values P(E | H1) and P( E | H2) are very small numbers. For example, we might get something like
  • P(E | H1) = 0.00000000000000000001  (10 to the minus 20)
  • P(E | H2) = 0.00000000000000000000000001  (10 to the minus 26)
For a statistician, the size of these numbers does not matter – we are only interested in the ratio (that is precisely what the LR is) and in the above example the LR is very large (one million) meaning that the evidence is a million times more likely to have been observed if H1 is true compared to H2. This seems to be overwhelming evidence that the suspect was a contributor. Case closed?

Apart from the communication problem in court of getting across what this all means (defence lawyers can and do exploit the very low probability of E given H1) and how it is computed, there is an underlying statistical problem with small likelihoods for non-exhaustive hypotheses and I will highlight the problem with two scenarios involving a simple urn example. Superficially, the scenarios seem identical. The first scenario causes no problem but the second one does. The concern is that it is not at all obvious that the DNA mixture problem always corresponds more closely to the first scenario than the second.

In both scenarios we assume the following:

There is an urn with 1000 balls – some of which are white. Suppose W is the (unknown) number of white balls. We have 2 hypotheses:
  • H1: W=100
  • H2:  W=90
We can draw a ball as many times as we like, note its colour and replace it (i.e. sample with replacement). We wish to use the evidence of 10,000 such samples.

Scenario 1: We draw 1001 white balls. In this case using standard statistical assumptions we calculate P(E | H1) = 0.013, P(E|H2) = 0.0000036. Both values are small but the LR is large, 3611, strongly favouring H1 over H2.

Scenario 2: We draw 1100 white balls. In this case P(E | H1) = 0.000057, P(E|H2) < 0.00000001. Again both values are very small but the LR is very large, strongly favouring of H1 over H2.

(note: in both cases we could have chosen a much larger sample and got truly tiny likelihoods but these values are sufficient to make the point).

So in what sense are these two scenarios fundamentally different and why is there a problem?

In scenario 1 not only does the conclusion favouring H1 make sense, but the actual number of balls drawn is very close to the expected number we would get if H1 were true (in fact, W=100 is the 'maximum likelihood estimate' for number of balls). So not only does the evidence point to H1 over H2, but also to H1 over any other hypothesis (and there are 1000 different hypotheses W=0, W=1, W=2 etc.).

In scenario 2 the evidence is actually even much more supportive of H1 over H2 than in scenario 1. But it is essentially meaningless because it is virtually certain that BOTH hypotheses are false.

So, returning to the DNA mixture example, it is certainly not sufficient to compare just two hypotheses. The LR of one million in favour of H1 over H2 may be hiding the fact that neither of these hypotheses is true. It is far better to identify as exhaustive a set of hypotheses as is realistically possible and then determine the individual likelihood value of each hypothesis. We can then identify the hypothesis with the highest likelihood value and consider its LR compared to each of the other hypotheses.

2 comments:

  1. Hi everyone. Am Yolanda , am here today to share my good experience with Dr Blessing with you all on this page and am very happy posting this because for Dr blessing of Blessingspiritualtemple@gmail.com has makes me a very happy woman today, i was very depressed and devastated when Maxwell my boyfriend of 2 years left me for another girl because some of his family members never likes me and this messed my life up in so many different ways , because Maxwell and i have been together for too long and we even making plans of getting married when this whole issues came up , and at this point i have did all i could to make he keep this our love life going , but all end up at nothing, and this kept on running on my head, at this time i couldn't focus on anything i lost my job, even at this i kept on praying to God to bring back my ex i contacted some spiritualist online they never make Maxwell love me again , this came up to two months now and i have been at the edge of giving up on this because nothing seems to be working out when i ran into a comment on Facebook about Dr blessing , even when i have wasted time and money with others i was moved by the post i read about this Dr Blessing so i said to give it my last trial , i contacted he on 9th of June 2017 and explained my situations , there he told me few instructions to follow and i did the things he said after which he said in 3 days from now i will hear or get message from Maxwell i never believed he because others has also told me same thing , it was like a dream on the 12th of June 9 34 AM Texas time this year when i received a very sweet text from Maxwell and i called to thank him and he said i should tell as many people i can able his work as many are called but few are chosen , so i want to tell you all here if there is anyone out here passing through hard time in his /her relationship, if your spouse cheat, if your partner is wanting a divorce for no good reasons , kindly contact Dr blessing today and be happy again in life,
    Email; blessingspiritualtemple@gmail.com
    Whats,App. +1(951) 409 0694

    ReplyDelete

  2. Real and Powerful Lottery and good luck spell email; ( efepowerfultemple@gmail.com ) for urgent help!!
    Hey guys, I'm so excited to share my TESTIMONY ON HOW Prophet EFE (GREAT SPELL CASTER) HELPED ME TO WIN A LOTTERY. I won EuroMillions Millionaire Raffle prize of 1,000,000E on March 2017, I took an advice from a friend who talked about this great spell caster on the Internet, He placed a testimony on a blog saying how this prophet helped him win the lottery by sending him the winning numbers , I was curious and I thought it was all joke not until I contacted this spell caster to know for myself how this works because I had spent a lots buying tickets and I never won. I contacted him and he told me the necessary things needed to be done and I did just as he said and he told me to wait for just 3 days, on the 3rd day he gave me the winning numbers to play the lottery which i did, Can you believe my name was the first among winners. He told me all i need you to do for me is make sure that you share this testimony to others in need of help so that they can also win the lottery. I am sharing this testimony with everyone who needs help to win lottery. Contact this great priest through his email address ( efepowerfultemple@gmail.com ),you can also call him on : +2347081602438 .contact him for money spell, Win Lottery spell, Business spell, marital problems spell, good luck in curt case spell, and many more.thank you Prophet Efe i own you my life.

    ReplyDelete