23andMe health reports - how accurate are they? 23andMe review part 1
23andMe, a new fab fad
The 23andMe test is currently one of the most famous DNA tests in the world and one of the most purchased ones. This test along with the likes of another website behemoth Ancestry.com, have helped alter public awareness about DNA testing. As we explore in the next two articles reviewing the 23andMe services, you will find out why they are so popular - by offering really smart (and practically addictive services) for an incredibly low price for what the consumer receives. You pay for the test, your kit arrives, you spit in a tube and in 3-5 weeks you get online results!
23andMe was originally started to ascertain ancestry information where the tests probed for select DNA sites in our genomes to help gauge the ethnic origins of an individual. But unlike its primary competitor Ancestry.com, 23andMe now can also offer health and wellness related information based on personalized genetic information testing. There is actually a plethora of such tests on the current market - some good, some bad - with some requiring a doctor's order while others do not. But be wary, many are inappropriate for the use that consumers have in mind.
The tests that do not require a doctor signing a requisition form are referred to as a direct-to-consumer DNA test. Typically, the lack of a physician’s consideration should be an automatic red flag in terms of the quality of test being offered as there are additional, minimal criteria that a DNA test should have to satisfy a physician. However, 23andMe is unique in that it actually satisfies many of the these criteria that would be expected from a clinical test and 23andMe has even obtained FDA-approval for testing for some designated conditions - the only such direct-to-consumer tests to have received such approval.
Having said that, a 23andMe test is NOT a clinical test!
As such there are serious limitations and many potential problems that could be associated with its use. But we'll get to the benefits and limitations of these tests shortly.
Recently, we wrote a series of articles about a Canadian family of three generations that has undergone their full genome sequencing to screen for information that could impact their medical management. This is a technologically progressive family that is no stranger to using advanced technologies or novel services to probe for information related to their health and as such some of the family members had previously screened themselves with 23andMe tests as well. This gave Merogenomics a perfect opportunity to compare and analyse the results between these two different approaches.
But before we get to the comparison of the data, we first have to look in detail at the 23andMe testing.
23andMe test make up
First of all, 23andMe is a behemoth website offering tons of services for your buck and in our opinion, one of the best direct-to-consumer services available. 23andMe services can be divided into five primary components by the types of results that you receive. There is the non-health related information that can be divided into ancestry information (an attempt of defining ethnic make-up of an individual) and trait information (this is fun and mostly obvious information that in and of itself does not seem to be very good marketing for the potential serious significance of DNA testing with traits like your eye color, viscosity of your ear wax, ice cream flavour preference, although it has definitely captured some people’s interest - the recent addition of predisposition to stretch marks.)
The health related portion of 23andMe tests can be divided into three components. The first one is wellness information which includes still experimental information related to weight, sleep patterns, certain dietary concerns and muscle type composition potentially related to training type. However, the majority of these are still not verified and some are based on 23andMe’s own predictive models using up to hundreds of variants (a polite way of saying mutations) to generate a polygenic risk scores. We will revisit this topic later in the next 23andMe review article.
The second and third health related components of 23andMe tests are referred to as “Health reports” and are the primary reason and focus of this two-part 23andMe review: they are the “Carrier Status” and “Genetic Health Risk” reports.
Carrier status refers to the genetic mutations that a person might carry that typically do not develop a condition but if such mutation is passed onto a child from both parents, the child can be at risk of developing a condition. We recently discussed the topic of DNA carrier status in detail when analysing the genetics of that first Canadian family that screened themselves for health predispositions with full genome sequence testing.
Genetic Health Risk reports (also referred to by 23andMe as “Health Predisposition” reports in case all of this was not confusing enough to you) are the most serious aspect of the 23andMe tests with the most critical consequences. This report is the result of assessing your DNA for mutations that are linked to variety of health problems - currently testing for 12 conditions. The Genetic Health Risk report also provides one unique polygenic risk score that we break down in the part 2 of these 23andMe reviews so in total, 13 conditions are assessed. Therefore, a person better be ready for the results that might be brought up, as some of them are for conditions that are incurable such as Alzheimer’s or Parkinson’s.
Overall, there are also serious limitations to consider in the both Health Risk reports and so consumers should be very well acquainted with those limitations before proceeding on with any results. We will break it all down, below.
How scientifically accurate are the 23andMe health reports?
According to 23andMe, all Carrier Status and Genetic Health Risk reports are based on scientifically based genetic associations between the selected DNA variants and disease outcome. The DNA mutations that are assessed for these health reports are only those that have multiple sources of published scientific support with no published contradictory evidence. Since contradictions are extremely common in science due to the complexity of biological observations, these are quite stringent criteria. It is almost surprising that so many variants can indeed be included in each test with such a selection criteria. As 23andMe’s white paper on the topic states, “the [scientific] evidence should establish a consensus that the variant has a meaningful and real effect on the condition of interest”.
Furthermore, the published information is manually curated to ensure that the information can appropriately be extended to the commercial tests offered by 23andMe, meaning, 23andMe’s own hired scientists and medical professionals analyzed the published evidence. This is very sophisticated operation and very few service providers of DNA testing can boast of such claim (and hence you will rarely witness it). If that was not enough, 23andMe also analyzed public databases of compiled observations related to the mutations they test for. As such, these tests should be highly accurate in terms of the information for which they attempt to probe. But where there might be certain issues is in terms of the ability to accurately uncover that information and that is the primary limitation of these tests that has to be clearly understood thus why they cannot be used for medical purposes. 23andMe makes that very clear throughout its online content. We also might have some reservations with regards to some of the mutations selected for analysis, as you will see in the part 2 article.
Who is the 23andMe health test for?
It took days of analysis of the 23andMe website and their report results to come to our conclusions here.
If you are a person who does not care about the health result and is willing to find out just by chance, then all the more power to you. Just understand what you are getting yourself into. There is no doubt that the 23andMe tests will always find people with the mutations they screen for, and some of these lives will literally be saved as a result of this. There is no problem with that if we are talking about incidental screening. Read more about potential benefits of 23andMe for incidental screening below.
However, 23andMe health tests should not be considered seriously to screen yourself for critical genetic predispositions if you have legitimate medical reasons to seek such genetic screening. What would those be? First and foremost would be family medical history suggestive of potential genetic condition. If you suspect this then you should talk to you physician to discuss your alternatives which should include a visit with a genetic counsellor to determine what could be available to you through public healthcare means. Or alternatively what could be available to you privately that could actually be used for subsequent medical management.
Another reason could be either yours or your dependant’s medical history that has not yet yielded a satisfactory clinical diagnosis. Once again, there are actual clinical DNA tests that could be available to you that your doctor could rely on for your medical management. And 23andMe is not one of them.
There are only couple reasons why 23andMe health test should be considered:
- Inability to afford more expensive options where results can be used for medical decision
If you have no other recourse at all, and you know that the 23andMe DNA test checks a condition that you are suspecting, while the test will not be definitive whether you get positive or negative result, at least a positive result will start pointing towards some potential evidence that might have to be probed later on.
- Their one of a kind type 2 diabetes polygenic risk score (more about in next 23andMe review)
In some forms of screening, in our opinion, 23andMe should never be considered under any circumstances! This includes cancer predisposition! If you are concerned about cancer in your family, 23andMe would be woefully inadequate to do you justice and you should seek better alternatives. 23andMe recently has started providing information related to cancer predisposition but it is extremely limited information. It is one thing if someone stumbles across results from 23andMe and had no care to actually screen themselves, but it is altogether different if someone actively wants to determine if they could be genetically predisposed to cancer development. Read about the limitations of the test below.
Benefits of screening with 23andMe DNA test
Limitations notwithstanding, if you choosing to randomly assess your genetics with 23andMe tests, there are some definite benefits that can be derived:
- Health risk screening that otherwise is not available through a doctor
Since 23andMe now counts its clients in millions of individuals, there will always be those who will accidentally stumble on potentially important news that will impact their medical management. However, be aware of massive limitations below. From screening of random population with clinical DNA testing, we already know how prevalent diagnosis can be without any prior indication for testing. We also know that many who could benefit from genetic screening will never get access to the publicly available DNA testing due to too stringent criteria.
- Carrier screening prior to starting family
These days hardly anyone screens themselves genetically prior to having children. But that will change in time as the society understands how powerful DNA screening can be in avoiding passing on diseases to their offspring. While it seems like a small random chance, the reality of some of the genetic conditions can be quite overwhelming, both emotionally and financially, for the affected families. DNA screening for carrier status now provides an option to mitigate these risks, and this is truly the number one reason why we would promote the use of 23andMe tests. This is the largest component of the Health reports, currently screening for 44 such potentially debilitating conditions and these tests are cheap enough that all expecting parents could consider this investment prior to conception.
- If a person has been adopted with no access to past biological family medical history
In such circumstance, DNA testing provides a basic glimpse of genetic health background for adoptees. Once again, be aware of limitations and we recommend a far more comprehensive test for such a purpose (see alternatives in the part 2 article).
However, we stress that in all of these circumstances, we would always advocate instead for different genetic tests that are more accurate and more comprehensive. The above benefits are listed if you just don’t care too much what test you are taking and simply are seeking to satisfy your curiosity.
Limitations of 23andMe health test
- Incomplete data analysis
This is the biggest problem with a 23andMe test in comparison to what clinical tests actually offer. 23andMe only tests for very specific mutations along preselected genes related to a health condition of interest. However, a mutation can be found anywhere along the gene or even outside of a gene and still impact the gene function and that could lead to a condition. The biggest risk of 23andMe for a potential customer is not understanding this distinction and thinking that they have undergone complete testing. In such a scenario a negative result, meaning no significant finding was uncovered, the customer could be under the wrong impression that they are in the clear, but they are actually not. It could be a false negative result!
One case in point is the contentious issue of 23andMe testing for some BRCA gene mutations that predispose people to increased risk of developing breast and ovarian cancers, among others. At the 2019 American College of Medical Genetics and Genomics annual meeting, data was presented demonstrating that among nearly 125 000 patients tested for BRCA mutations, 23andMe test missed nearly 90% of additional BRCA mutation carriers, and even 20% among the Ashkenazi Jewish descent, the ethnic group most frequently presenting the BRCA mutations that 23andMe investigates. Since this is the only such comparison analysis that has come to light so far, we cannot know what kind of numbers would be missed with any other 23andMe tests. In this case, consumers of the 23andMe test have to understand that if their results are negative, at least for BRCA genes, they could still be in that 90% of patients with mutations untested by 23andMe and are not out of the woods. Same goes for all of the other genes tested by 23andMe.
One way to illustrate it is with a graphic below.
This is by far the most important limitation of the 23andMe health test that a consumer needs to understand because ignorance of this information can put the consumer directly at risk.
Another way to illustrate this is with a screenshot from the 23andMe site itself demonstrating how many different BRCA genes mutations the 23andMe test covers compared to already known and verified mutations in these genes.
- Positive results will require further expensive confirmation
If any results are found, they will require confirmation with appropriate testing before any medical decisions can be made. If you are just taking 23andMe test for fun, then you will have to be prepared for the possibility that if you do obtain results and if you plan to take any action, these results will have to be confirmed and it might come at your own expense. Typically these confirmation tests are no longer cheap. This is an important argument for considering an alternative and more appropriate test in the first place which results can be used directly by a medical doctor.
- False results
While this can and does occur to some degree to any test in the world, it is the frequency and how such false results can arise that can dramatically differ between tests. 23andMe uses genotyping technology, meaning it seeks for specific mutations along the genome with probes that will recognize the site. The probe is itself made of DNA, and it is a tiny fragment of DNA with a specific complimentary sequence that allows it to interact with precise site in your own DNA isolated from the saliva sample. When the probe finds the consumer’s DNA with mutation of interest, and contacts it, it induces a chemical reaction that allows measurement that the interaction took place. But the match between the consumer’s own DNA and the probe has to be perfect or it will not work. So if mutation is present in your DNA, the probe that is sent to find it will work. If mutation is not present, then it works not so well. But what if the mutation of interest was there, but very close by was another mutation that also affected how the probe interacts with DNA? Now you have a problem and the results might not be found, meaning the probe did not bind to DNA to discover presence of a mutation because of a confounding secondary mutation nearby. The customer would be informed it is a negative result when in reality it is actually a positive result. 23andMe actually provides lists of such mutations for all of its Health report tests, but does not indicate how frequently this takes place.
- Test failure rate
Your DNA is isolated from your saliva sample, and saliva can sometimes be problematic. This can be due to actual quantity of available DNA that can be isolated from cells, but apparently even what you do with your mouth prior to sample collection can skew results. As a consequence the test simply can fail to provide results. According to 23andMe’s own data, this can be quite frequent. For the BRCA genes analysis, they reported over 7% failure rate! Another quoted number, what we presume to be the overall test failure rate was reported by 23andMe to be up to 3%. Those customers who fail test results will have one opportunity to obtain a test kit again to collect new sample. If that fails again, they will be refunded and left with no result. To help your cause, don’t stick anything in your mouth for at least 30 minutes prior to providing your sample.
- Ethnicity relevance
23andMe’s health reports focus on DNA mutations that are observed more frequently in some ethnicities over others, and hence people of certain ethnic background are more likely to benefit from the tests than others. You can see the breakdown of available tests per ethnicity below to determine how likely you are to receive some results, and you can see two groups stand out: Europeans and people of Ashkenazi Jewish descent. Other ethnicities are far more limited in what the health tests can offer them. However, if you are of a different ethnicity, it is still possible you could receive a result for a particular variant even it if is predominantly observed in different ethnic backgrounds.
- Potential lack of customer preparedness to receive a significant DNA test result indicating the predisposition to a serious health condition
Ironically this problem is perfectly exemplified in a 23andMe video of a customer discovering BRCA gene mutation indicative of increased risk of breast and ovarian cancer:
- Customer did not think of any potential consequences of test results, was not aware of the risks or implications, and did not do online research about the test prior to purchase
- Customer only performed research on the result significance after receiving the result
- The customer admitted she was required to purchase much more expensive test afterward that was also not considered prior to 23andMe test purchase
- The customer was not prepared to understand the significance of the severe invasive medical procedures to be performed as a consequence of the genetic results
While in this case the result allowed for important intervention, some results are for conditions with no cure or intervention available. While the customer has an option of selecting to not obtain results for certain incurable conditions, the explanation about this option is not immediately obvious on the 23andMe website and potentially it is unlikely to be seen and considered for its significance by many customers prior to purchase or even after when receiving the results.
- Transplant patients being ineligible
If you have a transplant, basically you take someone else’s genome inside your body. The resulting cells can be present throughout the body, making it very difficult to isolate own DNA that was present at birth (called germline DNA). This definitely includes saliva samples as well and transplant patients should not take 23andMe tests. Good luck finding that information on 23andMe website (it’s there)! In the US, for every 10 million individuals at least 100 haemopoietic stem-cell transplantations are expected. So for the current minimum estimate of 10 million 23andMe customers that have been reported on their site, you could expect at least 100 clients who might have taken the test and would receive inaccurate results. Such customers are eligible for a refund. Yeah, 23andMe can handle that.
In the Merogenomics’ second article reviewing the 23andMe health reports, we will discuss what we consider are the right and wrong steps undertaken by 23andMe tests, we will look in detail at one of our favourite services offered by 23andMe related to health predisposition outcomes, we will further discuss your alternatives for DNA health screenings and also suggest what doctors should do when encountering 23andMe health related DNA test results.
In the mean time, happy DNA sequencing!
This article has been produced by Merogenomics Inc. and edited by Jason Chouinard, BSc. Reproduction and reuse of any portion of this content requires Merogenomics Inc. permission and source acknowledgment. It is your responsibility to obtain additional permissions from the third party owners that might be cited by Merogenomics Inc. Merogenomics Inc. disclaims any responsibility for any use you make of content owned by third parties without their permission.
Products and Services Promoted by Merogenomics Inc.