Comments from Arthur Snyder on my Essay #5:
Neutron Activation

Background
   Dr. Arthur Snyder is a physicist at the Stanford Linear Accelerator. He has been interested in the JFK assassination for a few years, and particularly its scientific aspects. Someone apparently called his attention to my Essay #5, on neutron activation, and asked him to comment on it. He eventually produced the highly mathematical critique that is at the end of this handout. Through an intermediary (Stu Wexler, whose messages you have seen in "External Comments"), he asked that I post it. I replied that I am only posting external comments that relate directly to the work of our class, and that this refers to an essay that we have not yet considered and that was not posted for the class’s purposes. I also noted that no one in the class would be able to follow his math and statistics. Eventually, we agreed that if he were to produce set of nontechnical comments that explained the principles of his critique in terms that the class could understand, I would post the critique and comments under PSC482G. Having read his nontechnical comments, I fear that they are still above the class, and so I have appended my explanation of his explanation right after it (pace Art). Thus, you have three chances to get one, as they say on the URI basketball team. Here is Art’s nontechnical explanation of his critique.

***********************************

Ken:

Here is my attempt at an informal, non-technical summary of my critique of your NAA essay for the students in your JFK class.
    First, the students should be aware that as a critique, my note assumes they have read your essay. I don't define terms you have already used (such as "marble cake") but presume that the reader is already familiar with them.
    I consider your use of the 83% "in the pack" fraction as the probability that the evidentiary samples are "separable" to be incorrect for two reasons:

o 83% is the best estimate we can make of the "in the pack" fraction averaged over Guinn's test samples, but it does not apply to individual bullets such as CE399. In your marble cake model this fraction would be expected to vary substantially from bullet to bullet.
o Even if all Carcano bullets had a common 83% "in the pack" fraction this would not be the probability that a group of bullets would be "separable". For the case of only two bullets, the chance that both would be from the "separable" "in the pack" component would be 0.83 squared or about 0.7. Things are actually a little better than that since if both samples were from the minority (what I called the "flyer") component, they would be "separable" too.

I also question whether the bullets are separable even if all samples were from the "in the pack" component. The number of accidental matches between evidentiary samples and Guinn's test samples indicates that the chance of an accidental match is not all that small.
    To compare the hypothesis that CE399 caused Connally's wrist wound to the hypothesis that it was caused by a different (Carcano) bullet one needs to compute the likelihood for each hypothesis to produce the observed match. The ratio of these two likelihoods indicates how much the data prefers one hypothesis over the other. Guinn's data on his 3 sample bullets is insufficient to estimate this ratio accurately; as best I can tell it is roughly 4/1 in favor of the wrist fragment coming from CE399. This is not overwhelming evidence for the single bullet theory.
    Quite apart from the origin of your 83% the extension to claim that "there is an 83% probability that the hospital bullet had first passed through Connally's wrist, that all the other fragments came from a single head shot, and that only those two bullets-both fired from Lee Harvey Oswald's Mannlicher-Carcano rifle to the exclusion of all other rifles-hit Kennedy and Connally that day" is not correct. Since we don't know the a priori probability of such events, there is no way to calculate such an absolute probability.
    The only thing we can hope to calculate are the conditional probabilities. That is:

o The probability that IF CE399 was the source of the wrist fragment, a match would be observed.
o The probability that IF some other bullet was the source of the wrist fragment, it would match CE399 accidentally.

Basically, what I'm objecting to is the quantification of absolute claims. If the likelihood ratio were much larger (or much smaller than one), then one could be fairly confident in selecting one hypothesis over the other. It is not possible to calculate a probability that such and such occurred in unique situations for which the a priori probabilities can't be known.
    The only firm conclusion (and it is an important one) that can be drawn from Guinn's measurements is that there is no evidence of anything other than Western Cartridge Carcano bullets among the fragments in evidence. Thus, the data is consistent with the single bullet theory, but does not prove it.
    Finally, in addition to the critique I offer some suggestions about how the situation could be improved. As far as I can tell the results could still come out either way or remain ambiguous. Unfortunately, we are unlikely to get permission to extensively sample CE399 or to find the money that would be needed to perform the extensive NAA analysis required on it as well as samples of other bullets.

Arthur

******************************************

Comments on the Statistical Analysis in Ken Rahn's Essay:
"Neutron-Activation Analysis and the John F. Kennedy Assassination"

Arthur Snyder

Ken Rahn in his essay Neutron-Activation Analysis and the John F. Kennedy Assassination makes several errors in his statistical analysis. These errors invalidate his conclusions. In this note I will elucidate the nature of these errors and demonstrate how to properly compare the hypothesis that the fragments in Connally's wrist are from CE 399 (the "magic" bullet) to the hypothesis that the Connally fragments are from a different bullet.
    The relevant portion of Rahn's essay is quoted below:

    Since Guinn’s data on heterogeneity are the only ones we have, we must base all our conclusions about separability of groups on them. What do his data show? That of 12 zones within bullets, only two differed markedly from the "pack." The other ten zones varied from one another by only 8% on the average—about the same as Guinn’s anecdotal 6%. Thus for Mannlicher-Carcano bullets as a whole, 83% of the time (10 of 12 cases) we may expect bullets to produce highly reproducible fragments (heterogeneities of only 8%), and 17% of the time (2 of 12 cases) we will find much larger variations. Since variations of 8% will keep Guinn’s two groups of fragments separate to a very high confidence limit (between 95% and 99%), we may conclude that there is an 83% chance that the two groups are actually separate to the 95–99% confidence level, and a 17% chance that they are not separate. This is the probabilistic result referred to earlier.
   How should we view an 83% probability of having separate groups, strong support for the SBT, and all those other good things? Obviously, it’s not the 95% probability that a scientist would like, nor the 90–95% that a jury needs in order to convict in a criminal trial. But it’s stronger than any other evidence for the SBT, and far better than any evidence against it. And so it controls the issue. Maybe some day someone will analyze a much larger suite of MC bullets for heterogeneity. Maybe Dr. Guinn will release his large set of analyses from so long ago. In either case, we could have more confidence in our probabilities. But until then, we may state that there is an 83% probability that the hospital bullet had first passed through Connally’s wrist, that all the other fragments came from a single head shot, and that only those two bullets—both fired from Lee Harvey Oswald’s Mannlicher-Carcano rifle to the exclusion of all other rifles—hit Kennedy and Connally that day.

The table at end of this note contains the measurements Ken is referring to. The measurements are the antimony content (in parts per million (PPM)) for four measurements Guinn repeated on each of five bullets. The first 3 bullets in the table represent studies of heterogeneity, i.e., the same bullet sampled multiple times. The last two are studies of measurement reproducibility, i.e., the same sample from each bullet measured multiple times. As Ken notes, the measurements from three bullets studied for heterogeneity seem to cluster, e.g., bullet 6002A has three measurements ~900 and one flyer way down at 358. 6001B yielded three measurements of ~ 400 and one flyer way up at 667. The choice of what is a flyer is a bit arbitrary. Qualitatively at least his division is not unreasonable—2 flyers and 10 "in the pack."
    Rahn's 83% is the fraction of Guinn's measurements that are "in the pack"—"10 out of 12 cases". His application of this "in the pack" fraction to CE 399 implicitly assumes that this same fraction applies to it. However, 83% is really only the best estimate of this fraction averaged over Guinn's samples; there is no justification for assuming it is the same for each bullet Guinn sampled much less CE 399. In the "marble cake" model Ken proposes for Carcano bullet variations one would expect that the number (there need not be just two) and sizes of regions of differing composition to vary from bullet to bullet. The probability that a bullet which was a 50/50 mixture of a component with an antimony content ~350 PPM and a component of ~900 PPM would produce the 1/3 pattern observed for 6002A is only 2/3 times smaller than the probability of the most probable 2/2 pattern. Thus, from the extremely limited statistics available there is no evidence that the ~350 PPM "flyer" component of 6002A is substantially smaller the ~900 PPM "in the pack" component. Using the average 83% average "pack" fraction for 6002A or for CE 399 is not justified.
    For the sake of argument let us assume that for all Carcano bullets, 83% of the lead has one antimony content and the other 17% has another significantly different antimony content. Even in this case 83% is not the probability that two samples from the same bullet would match. In the "marble cake" model in which two samples from the "flyer" component would also expected to be near each other the probability of getting a match is given by:

P_match=CL x (F_p² + F_f²)

where CL is the confidence level of the match considered to be acceptable, F_p is the "pack" fraction and F_f (=1-F_p) is the "flyer" fraction. For a 90% confidence level and our assumed 83% "in the pack" fraction P_match is 0.64. This does NOT mean that there is an 64% chance that CE 399 caused the damage to Connally's wrist; but only that if CE 399 did leave the fragments in Connally's wrist, there was a 64% chance that Guinn would have found a match and 36% chance that he would not have seen a match. Even if CE 399 had not matched the wrist fragment, that would not have been strong evidence that the wound was caused by a different bullet [a]. The fact that it did match is not strong evidence that CE 399 caused the wound either.
    To establish that CE 399 caused Connally's wound one needs to show that the probability of an accidental match between two different bullets is much smaller than the probability of a self-match. Guinn actually claimed that this is the case in his HSCA testimony, but it is not true, e.g., the Connally wrist fragment at 797 PPM using a 150 PPM tolerance (which matches ~90% of the "pack" measurements from Guinn's test samples) matches 6002A samples 3 and 4, 6003A sample 1 and measurement 4 of 6001B. The three fragments supposed to have come from the head shot with antimony content ~620 make a good match to 6003A sample 1 and to 3 out of 4 of 6001B repeated measurements. Presumably, nobody is going to claim that Guinn's 6002A caused Connally's wounds or the 6001B was used for the head-shot [b]!
    Based on these accidental matches we can roughly estimate that the probability of an accidental match is ~1/5 in which case the ratio of the likelihood of a self-match to an accidental is ~4/1. There is not enough data to get an accurate estimate of the probability of accidental matches, but it is clear from the number that occur between the wrist fragment and Guinn's sample bullets that an accidental match is not "extremely unlikely, or very improbable" as Dr. Guinn claimed in his HSCA testimony.
    Quite apart from the origin of Ken's 83%, the claim that "there is an 83% probability that the hospital bullet had first passed through Connally’s wrist, that all the other fragments came from a single head shot, and that only those two bullets—both fired from Lee Harvey Oswald’s Mannlicher-Carcano rifle to the exclusion of all other rifles—hit Kennedy and Connally that day" is a statistical non sequitur. Without knowing the a priori probability of these events such a probability cannot be computed. The a priori probability Guinn's 6001B being used during the assassination is clearly zero. The probability that 6001B was used for the head shot is ZERO despite the fact that the odds were something like 4/1 against the fragments found in the car and recovered during the autopsy matching it.
    Rahn's claim is similar to the nonsensical assertion made by the HSCA's acoustic experts that there was a 95% chance there was another shooter on the grassy knoll. Statistical arguments can not deliver such grand claims.
    The situation could be improved if multiple precision measurements could be made of lead from the base of CE 399 [c]. Since, only lead from the base is supposed to have been deposited in Connally's wrist, the variations in antimony over the whole bullet are not relevant. It can be hoped that the variation of CE 399 base lead is substantially smaller than the variations seen in Guinn's samples. This would increase the probability of a self-match and decrease the probability of an accidental match. If the antimony content distribution of base lead from CE 399 turned out to be narrow enough and centered near the current measured value of 833 PPM [d], the results could even turn out to be in conflict with the single bullet theory.
    With the currently available data we cannot discriminate between CE 399 causing Connally's wrist wound and the wound being caused by a different Carcano bullet that accidentally happened to have similar antimony content. However, "lone nutters" can take comfort from the fact that there is no evidence of anything other than Western Cartridge Carcano bullets. Most other bullets have antimony contents at the several percent level and would have been easily detected if they were among the fragments tested. Thus, while Guinn's measurements do not establish the SBT, they are consistent with it.

Intra-bullet variations measured by Dr. Guinn

Notes:

    [a] Unless the fraction had been completely outside the ~1200 PPM upper limit on Carcano antimony content—indicating a totally different type of bullet.
    [b] Curiously, another "match" occurs between the bullet removed from General Walker's wall and the unfired round found in the Carcano rifle found on the 6th floor of the Texas School Book Depository. Guinn in the report he submitted to the HSCA (not his testimony) uses this match to establish that the Walker bullet is from a Carcano!
    [c] If such a program of measurements could be undertaken it would be important to carefully characterize the antimony distribution for unrelated sample bullets like Guinn's with high precision measurements and adequate statistics. Nearby samples need to be compared to confirm the "marble cake" model. The protocol for establishing a match should be developed using data on the samples before the data on CE 399 is examined in order to avoid pernicious ex post facto biases, e.g., if CE 399 had measured 646 (like the not-quite matches of 6001B) instead of 833 PPM, this would probably have been considered a good enough match. In the unlikely event that the opportunity to make these measurements should arise, they should be done blind, so that which fragments come from which bullets is not known to the experimenters till after measurement and matching are complete.
    [d] I don't know where Guinn sampled CE 399. Probably not the base as he had no reason to expect the inhomogeneity that afflicts these bullets. A sample from the base might turn out to be quite different from the one Guinn measured.

Back to NAA