23andMe health reports - how accurate are they? 23andMe review part 2
This is a second article of two dedicated to reviewing 23andMe DNA based health reports. The first article focused on what constitutes the 23andMe health reports, who could be considering the 23andMe health testing and what are the major test limitations to consider prior to testing. We continue the 23andMe review with what impresses us about their process, what does not, and where should medical doctors stand when it comes to 23andMe results.
What 23andMe is doing right
- Tell you to seek a genetic counsellor
This is good because every genetic counsellor would likely inform you to stay away from this type of test for serious health-related screening or diagnostic testing and instead point you to appropriate alternatives based on your actual clinical needs. Be aware that the Canadian customers are not sent to the Canadian database of genetic counsellors but rather the American one. Further, this does not solve the issue of whether, under the public care system in Canada, the clients could actually be able to review the direct-to-consumer results with a local genetic counsellor without referral.
- Warn you that this test should not be used for medical decisions
This is by far the most important warning that 23andMe can make, for the benefit of both consumers and the doctors to whom 23andMe is so eager to send customers. This warning is present on the homepage itself, but does not jump out at you, which it should. As you dig little deeper through the site, there are plenty of reminders which is good, although again, they all should be even more explicit.
- Quality of the testing is rigorous so the information obtained is highly likely to be accurate
We are talking about testing done in a certified laboratory where each result has been meticulously (to varying degrees) tested for accuracy of the test and in fact, it is the only such direct-to-consumer testing (not requiring a doctor for a test order) that is FDA approved on the market. This means that a positive result can be taken seriously enough that confirmation testing should be sought.
- Provide the customer with an option to not receive certain genetic health risk results
The customer can opt out of obtaining information for serious conditions that have either no intervention or no cure, or conditions with a limited likelihood of being discovered while potentially eliciting great deal of anxiety while waiting for the results. This includes BRCA breast/ovarian/prostate cancer predisposition information, MUTYH colon cancer predisposition, as well as Alzheimer’s and Parkinson’s genetic predisposition. Not everyone would wish to receive such information knowing that exposure to such data could cause severe stress and anxiety.
- The personal risk for a genetic condition is provided to the customer as an absolute risk
When possible, 23andMe attempts to frame the potential risk of genetic result as an absolute risk in contrast to a relative risk. Absolute risk is your personal risk. Relative risk is in comparison to another group, for example, smokers vs. non-smokers. Therefore providing absolute risk is more personally meaningful. In addition, when made possible, this risk is also provided for a specific ethnicity when it is not necessarily comparable to risk of a different ethnicity. That is some impressive scientific digging!
- Using incredibly large number of customers to drive scientific medical research
The number of customers listed on 23andMe website now exceeds 10 million. This is an enormous database worthy of a population of some entire nations. Some of the results collected from customers has allowed for commercial genetic analysis that is unique to 23andMe (such as type 2 diabetes risk score. See below).
- High level of customer data protection and data autonomy
Your data appears to be in very safe hands and it is very unlikely that identifiable individual genetic information would become available to anyone. It appears highly unlikely that someone’s genetic data could be abused in some manner. All individualized analysis of customer information, including genetic information, has to be approved by customer and this would likely occur only under very unique research circumstances. Otherwise individual customer data is deidentified and analyzed in aggregate while pooled with all the other customers together. Researchers can fish out statistics of what is observed in a population as a whole, but not in what is found in a particular person’s data.
- The customer owns their DNA data and all information can be requested to be destroyed
Sounds good, right? Although there is the caveat of how long the information needs to be stored due to regulations governing the CLIA laboratory certification process. This is the highest level of laboratory certification aiming to analyze human samples available in the US. For that reason 23andMe appears to be required to hold your DNA data for two years after your request for data deletion. That is as per CLIA requirements.
But 23andMe also mentions it is also required to abide by the CAP accreditation requirements (one of the most prestigious accrediting bodies providing the CLIA certification and which can impose its own additional regulations). Now that could be a very different time frame altogether and who knows what that might be – although one source suggested 10 years data retention! It would be great if 23andMe was clearer about this information on its site and could mention how long customer DNA data is actually retained. But theoretically, once you request your data to be erased, the majority of your information will be destroyed and your data will no longer be used for any research purposes.
- Great support information for customers
We highly recommend reading the 23andMe website thoroughly as it has heaps of important and educational content of value, irrespective of what test you end up taking in the end. Sheer joy for nerds!
- Forces the customer to take educational tutorials to raise awareness of the impact of their results
This includes stating that results are not diagnostic and how this information could impact them or other family members emotionally from the effects of receiving unwelcome information. This serves as the last chance warning for those who were not prepared when purchasing the test.
What 23andMe is doing wrong
- Risks are not made more explicit
Some of the details about the risks of testing are buried, especially as they apply to health related information which should be taken seriously. The list of limitations is long and you can see it the previous 23andMe review dedicated to its health reports. One example is mentioning the potential psychosocial implications as a result of experiencing greater anxiety from obtaining unexpected results not only as it relates to personal health, but also potentially regarding family relations that might turn out to be different than what the person was lead to believe (member of a family turns out not to be a biological family member). Some of these warnings might not be encountered by the customer unless they carefully read the terms of agreement. Many will miss these important points.
- Stating that the 23andMe tests meet clinical validity
This is not appropriate as it can create confusion. The results are not meant to be used for medical decisions and must be confirmed with “additional testing in a clinical setting”. Making a statement that their tests meet “clinical validity” is definitely a gray area because they based this solely on the variants they selected being validated in scientific reports and some clinical guidelines as potentially causative of the conditions being tested for. This could insinuate that the 23andMe test itself is clinically valid, but if it was then further confirmation would not be necessary. Using such language can lead one to believe that the test has direct medical validity and can confuse both the client and the physician who will not have time to wade through pages of information to find disclaimers. This confusion can be further compounded by the fact that tests are also advertised as FDA-approved which normally is reserved for diagnostic testing.
- No explicit warning is provided with regards to the ethics of testing children
Many of the conditions tested for in the Genetic Health Risk portion of the 23andMe health test do not develop until adulthood, have low penetrance (meaning small likelihood of actually materializing), would never be considered for testing in children under clinical conditions, and are not supported by clinical guidelines on ethical grounds. While 23andMe does have a great site assisting parents’ conversation with children with regards to DNA testing that we would recommend for everyone to read (children or not), this information is buried too deeply and no professional stance is taken on testing children for adult onset conditions. This warning should be overtly stated.
- Appropriate testing alternatives are not listed at all!
Not good. Consumers should be informed of appropriate alternatives that overcome the limitations of 23andMe tests. Merogenomics will do it instead, see below.
- The customer has no say in terms of how their de-identified data is being used for analysis by undisclosed third parties
Customers’ results are used for data mining and this is likely a primary profit source for 23andMe. The customer has no say in that at all. Did you even know that 23andMe participates in drug development? Which companies or institutions are currently involved is also not mentioned (although some past examples are cited).
- Clients are deferred to family physicians or genetic counsellors for any remaining gaps of knowledge or further questions
Only in one instance that we could find, buried somewhere very deep, did 23andMe actually specify that it should be physicians with knowledge of genetics that should be contacted. Otherwise, all over their site, you are politely requested to contact your physician if you have any concerns. This is ridiculous! Sending 23andMe customers to unprepared physicians to fill these gaps of knowledge is inappropriate as it is unlikely that physicians will have had the luxury to study the 23andMe website and inform you of what 23andMe actually had in mind. Rather, 23andMe should employ their own medical professionals to be able to fill these gaps in knowledge and provide appropriate recommendations for any further steps, including referrals to specialists based on any pathogenic findings.
- Encouraging clients to discuss their positive results with family members
This would only be appropriate if the results of the tests were confirmed to be accurate. Otherwise it might lead to unnecessary worry and cascade testing. One can argue that discovery of positive results in other family members helps to confirm the test result but once again, any person who uncovers pathogenic variants through 23andMe will still have to undergo additional clinical testing to confirm the result which means additional expense. Therefore we believe that the results should be confirmed prior to propagating the information (that might be unfounded) through the family.
23andMe type 2 diabetes predictive score
As of early 2019, 23andMe offers a one of a kind type 2 diabetes predisposition polygenic risk score. It is called polygenic because it assesses literally hundreds of variants (science’s preferred term for mutations) along the DNA to determine overall risk. This is by far one of the coolest features of the 23andMe test because it stands out from anything else in the DNA world.
23andMe builds a very compelling case, based on past published scientific literature, in its White Paper dedicated to the topic of testing for prediabetes. While it is estimated that more than 70% of people with prediabetes will end up developing type 2 diabetes, only 12% of prediabetics are actually diagnosed. This is very significant because diabetes is exceedingly expensive to treat (with a quarter of the entire US healthcare expenditure believed to be siphoned off for the treatment of diabetes and its complications), while a number of interventions are currently available that can reduce the risk of progression from prediabetes to full blown diabetes. The simplest form of intervention is lifestyle changes including exercise or weight loss which can lower the risk of type 2 diabetes by more than 50%. In addition, a number of drugs have been shown to reduce the risk of progression from prediabetes to diabetes as well.
One revealing aspect of 23andMe’s White Paper to demonstrate the scientific validity of their prediabetes polygenic risk score was how many sites of your genome are actually investigated for mutations. The number is quite staggering! If you count them all together, it amounts to 1.7 million sites. Considering that all of our DNAs are at least 99.8% identical, with that many variants being assessed, 23andMe is practically getting your genome sequence without sequencing your genome.
For the development of its polygenic risk score model to estimate the individual likelihood of developing type 2 diabetes, over 2.5 million 23andMe customers who consented to participate in research were asked to respond to numerous health and lifestyle related surveys. This is where the power of 23andMe research lays, as the company has a vast pool of customers to draw from, and participation in research is hugely popular. Such a large dataset is what academic scientists fantasize about at night when they ponder their experimental designs but such large datasets are hard to come by in the published literature (albeit they are becoming more common in our digitalized age of big data collection). In essence, it is a very comprehensive genome-wide association study, where specific mutations along the genome are sought that are commonly found in those diagnosed with type 2 diabetes in comparison to those individuals who are not diagnosed. Two different models were developed, depending on how the DNA data was collected, with one of the models relying on 1244 variants across the genome. That is a lot of mutations compiled together that jointly can commence influencing one’s predisposition to the development of diabetes and it is certainly indicative of how complicated our genetic programming can be on our biology.
Based on user testing, 23andMe learned that the easiest concept for customers to understand their risk was based on remaining lifetime risk. In addition, since type 2 diabetes is a complex disease with root causes that are both genetic and environmental, including those that are under direct control such as diet and exercise habits, 23andMe has also developed a “prevalence explorer tool”. This tool allows the customer to view the effect of non-genetic factors on the prevalence of type 2 diabetes based on customer data with similar genetics but different age, BMI, diet, or exercise habits. This is probably the most incredible tool ever designed to aid motivating an individual to alter their lifestyle to improve their health based on learning their personal genetic risk. Finally, 23andMe provides a binary "increased likelihood" or "typical likelihood" result which is a genetic risk of type 2 diabetes development that is higher than the risk of developing diabetes due to being overweight, the most commonly suggested clinical guideline to screen for diabetes development. 23andMe states that 22% of the research participants would fall into this category, underscoring the potential value of this risk score.
While 23andMe clearly states that the type 2 diabetes risk score cannot be used for medical decisions (although hinting of a future possibility of being used for clinical decision making as more data is tested), it is one of the best commercially available polygenic risk score tools to encourage adopting healthier lifestyle habits that we know of (and the only one of its kind for type 2 diabetes). Considering the prevalence of this condition, currently regarded as an epidemic, it will be very interesting to see what the long-term impact of this tool will be, and potentially, this alone is worth the price tag of the 23andMe investment, if you elect to use this test (despite the limitations listed in the previous 23andMe health test review). Plus you are not left in the dark as 23andMe provides you with information on how you can act on your own to lower your risk of type 2 diabetes development if your genetics are not in your favour.
Doctors and 23andMe results
How should doctors react to 23andMe DNA health test results? Many of them might not have much experience with such test reports, but many more will likely be encountering them as the number of 23andMe customers seems to be growing exponentially. This particular section is specifically for physicians that might wonder how such data should be handled.
- Never disregard the 23andMe genetic health risk test results!
These are FDA-approved tests performed in CLIA-certified laboratories under stringent quality control standards and the result is very likely to be real. The test result should be taken seriously and must be verified. Send the patient for genetic counselling and for approved confirmatory genetic testing. Failure to do so might expose you to potential liability if negative outcomes do occur to a patient as a consequence of the uncovered genetic factors.
- Never use 23andMe test results for any medical decisions!
While 23andMe tests aim to follow high quality protocols of performance, the tests are not validated to be a clinical result and false positive can exists. Always confirm the results of 23andMe tests and prior to taking any further medical management action use approved confirmatory genetic testing and send the patient for genetic counselling. Failure to do so might expose you to potential liability if a patient is placed in unnecessary harm because of any medical procedures and the 23andMe test results turned out to be a false positive.
- Never encourage patients to use 23andMe tests for cancer predisposition screening!
The test is woefully inadequate to perform this function as only three variants are assessed for the BRCA genes and only two for the MUTYH gene and the accuracy of the result cannot be guaranteed. Not only are many additional candidate genes are left out, even the genes covered are not actually assessed in their entirety, leaving room for many unidentified true positives. Using 23andMe tests for cancer screening could lead to false negative results which could lead to lack of screening or taking prophylactic measures, thus jeopardizing patient safety.
Never encourage patients to take 23andMe tests if personal and family medical history warrants genetic testing The tests are not comprehensive enough to be used for diagnostic purposes and the accuracy of any result cannot be guaranteed. Instead, send the patient for genetic counselling and for approved confirmatory genetic testing.
- Never use 23andMe type 2 diabetes polygenic risk score result for medical management decisions
This component of 23andMe test is not FDA-approved and should be considered educational only. Inform the patient that the result is for educational purposes only and that the results currently cannot be confirmed as no alternate approved DNA based methods exist. Inform the patient that she or he is encouraged to adopt healthy lifestyle habits that can minimize the chance of type 2 diabetes development.
- For the carrier status results, inform the patient when it might impact reproductive decisions
Inform the patient with 23andMe results that both partners planning to have their own biological children would have to be matching in their carrier status for the same genes in order to undertake any reproductive decisions. If this happens, their children will have 25% chance of developing the condition. Send the expecting parents for genetic counselling and for an approved confirmatory genetic testing. You can also inform the patient that in some rare circumstances being a DNA mutation carrier can also result in a phenotype directly in the patient. If you observe the corresponding phenotype in the carrier patient, send the patient for genetic counselling.
Alternatives to 23andMe testing
The appropriate alternative to a 23andMe test is a DNA test that provides clinically valid results that a doctor can use directly for medical management. Merogenomics promoted DNA tests fall into this category. While these DNA tests do require a physician’s authorization they should also be available as a private, out of pocket expense if you do not meet the criteria for public test eligibility (where your healthcare coverage picks up the bill.) You can attempt to seek out these tests online on your own, but we do not recommend it unless you know how to differentiate a good test from a mediocre test from a total garbage test. A local genetic counsellor will be your best bet for advice but before you even consider that, first make sure to talk to your doctor about why you are seeking a test. You might have completely legitimate medical needs and might require a visit with a genetic counsellor anyway and they are the best for advice because clinical DNA tests are the type of tests they deal with exclusively.
What makes the test clinical for the doctor’s use? The accuracy and thoroughness of DNA information that is being investigated as well as the ability to clinically interpret the results. In terms of accuracy and thoroughness, this means that basically all clinical tests will be DNA sequencing tests and not genotyping tests. DNA sequencing ensures that every nucleotide will be decoded in the gene of interest. Specific forms of genotyping can still take place but only in regards to recognizing much more complicated structural DNA variants that are never considered with a 23andMe test. In terms of the data interpretation, this is now one of the biggest factors in differentiating the quality of DNA medical testing as the world of medical genomics is still evolving and the skill of interpretation can differ substantially between different certified laboratories. Some teams are definitely better than others.
Concerning medical DNA tests, depending on the level of information desired (which is based on what is scrutinized) and the financial resources available for testing, there are three types of tests that could be considered as alternatives to 23andMe tests.
- Targeted gene panels
This type of test looks at specific, preselected genes related to a condition for which the doctor will want to probe. These are by far the most common types of tests in the medical world and usually the most appropriate for the specific needs of the patient. A good example here is the cancer predisposition gene panel DNA test that will analyze only the genes that have been clinically validated in the past to be involved in cancer development, and these selected genes are investigated thoroughly for any changes. And even in this category itself, there can be a stratification of tests based on their capabilities. As mentioned above, tests will be differentiated by completeness of investigated information and quality of analysis. Unfortunately, the quality only rises with increased cost so cheap tests should always require a second scrutiny, just in case.
Such gene panels exist for pretty much every type of condition being screened by 23andMe with varying numbers of genes that can be investigated.
- Exome test
This could be considered the ultimate gene panel as it aims to include all of the genes in the human genome. It would be rare that such a test would have to be sought as an alternative to 23andMe since exome testing is used for attempting to diagnose conditions of unknown etiology where all other clinical diagnostic attempts have failed. We list it here primarily to differentiate it against the next test on the list.
- Full genome sequencing
This test analyses all of the patient’s DNA and is the true ultimate DNA test because it captures nearly all of the information possible in one single test. We are biased towards this test for that simple reason but unfortunately such a test will rarely be covered by healthcare or insurance unless it is medically needed (for similar reasons as mentioned above for the exome test) and therefore it can only be considered when financial means make it allowable. If it is feasible, then such testing allows for a single DNA test in your lifetime where the results can be analyzed for the rest of one’s life. 23andMe captures an insane amount of information, but whatever it does not detect there is a gap of knowledge that cannot ever be used for the future understanding of the DNA’s influence on health. A full genome, in theory, will also allow for captured DNA data to be analyzed in the future with more advanced technologies. For a list of benefits of full genome sequencing you can see our past post.
In the end, the type of test the consumer should be using is summarized best by the 23andMe website itself: “For people with specific needs (undiagnosed medical issues, for example, or intense curiosity and a generous budget), sequencing will become a more and more attractive option.” We could not agree more!
Comparing 23andMe to full genome clinical report
Let us conclude this article by comparing the 23andMe test results to a full genome test for one of the participants of the genome sequencing of a Canadian family that we wrote about in earlier posts. In one of the articles we recounted that this person was a carrier of the HFE gene mutation. 23andMe tests also investigates for same mutation in its health predisposition reports and has identified the same mutation as was reported to the doctor in the medical full genome sequencing report. We have to admit, 23andMe’s health report interface is truly well designed and beautifully presented. Complex science is delivered in a logical pattern, with thorough explanation of the meaning of the test results and potential next steps. It was very impressive work!
However, the 23andMe test also provided a result indicating that this individual was a carrier of A69S variant in the ARMS2 gene. This was not reported at all to the doctor in the medical report resulting from full genome sequencing.
The full genome sequencing test also identified the heterozygous A69S variant in that gene. So why would it not be reported in a medical report? The company was kind enough to provide detailed explanation. The reason why it was not was reported because it was deemed there was not enough evidence of pathogenicity to bring it to the American College of Medical Genetics and Genomics criteria that are used as the current gold standard across laboratories to assign a variant with level of being pathogenic or likely pathogenic. These are stringent criteria that determine what the cut off is to report to a doctor for both the disease risk and carrier status, in an attempt to ensure consistency in medical reporting of DNA data interpretation.
There were two main points which brought its score down. The biggest is that this particular variant has a very high population frequency. Looking at one of the databases that tracks this information, the GnomAD, the average frequency is 0.25 but it goes as high as 0.42 in some populations –meaning that in some populations 42% of people can be carriers. This argues strongly that it is not a directly disease-causing variant, but a common polymorphism. The second is that there is evidence in the literature of a statistically significant correlation between this variant and age-related macular degeneration, but not a clear causal relationship given the number of individuals who carry the polymorphism but do not show evidence of the disorder.
This small example perfectly illustrates the difference between a non-medical DNA test - that simply screens for information without deep analysis of the result - and an actual, sophisticated test that aims to provide the DNA test authorizing physician with meaningful data towards patient medical management. It almost makes you wonder why 23andMe has even selected this variant for reporting considering that its own data on variant frequency among its own clients is similar to what is stated above (and even higher), thus one of the eliminating criteria for why the variant is not appropriate for reporting. In this case, it only runs the risk of causing undue stress on a test recipient.
Overall, we are very impressed with the type of work that 23andMe is doing and we hope to see more amazing research coming from the company. However, we would not recommend 23andMe tests for health risk predisposition testing but rather we would recommend considering those funds being diverted to clinically meaningful testing instead. In circumstances where this is not possible due to financial constrains, or simply because the actual results are not immediately important to you, 23andMe can offer some good advantages over no DNA testing at all, and potentially provide some fun along the way.
So in other words, happy DNA sequencing! But make sure of that!
This article has been produced by Merogenomics Inc. and edited by Jason Chouinard, BSc. Reproduction and reuse of any portion of this content requires Merogenomics Inc. permission and source acknowledgment. It is your responsibility to obtain additional permissions from the third party owners that might be cited by Merogenomics Inc. Merogenomics Inc. disclaims any responsibility for any use you make of content owned by third parties without their permission.
Products and Services Promoted by Merogenomics Inc.