Societal preferences for the return of incidental findings from clinical genomic sequencing: a discrete-choice experiment
=========================================================================================================================

* Dean A. Regier
* Stuart J. Peacock
* Reka Pataky
* Kimberly van der Hoek
* Gail P. Jarvik
* Jeffrey Hoch
* David Veenstra

## Abstract

**Background:** An important challenge with the application of next-generation sequencing technology is the possibility of uncovering incidental genomic findings. A paucity of evidence on personal utility for incidental findings has hindered clinical guidelines. Our objective was to estimate personal utility for complex information derived from incidental genomic findings.

**Methods:** We used a discrete-choice experiment to evaluate participants’ personal utility for the following attributes: disease penetrance, disease treatability, disease severity, carrier status and cost. Study participants were drawn from the Canadian public. We analyzed the data with a mixed logit model.

**Results:** In total, 1200 participants completed our questionnaire (available in English and French). Participants valued receiving information about high-penetrance disorders but expressed disutility for receiving information on low-penetrance disorders. The average willingness to pay was $445 (95% confidence interval [CI] $322–$567) to receive incidental findings in a scenario where clinicians returned information about high-penetrance, medically treatable disorders, but only 66% of participants (95% CI 63%–71%) indicated that they would choose to receive information in that scenario. On average, participants placed an important value ($725, 95% CI $600–$850) on having a choice about what type of findings they would receive, including receipt of information about high-penetrance, treatable disorders or receipt of information about high-penetrance disorders with or without available treatment. The predicted uptake of that scenario was 76% (95% CI 72%–79%).

**Interpretation:** Most participants valued receiving incidental findings, but personal utility depended on the type of finding, and not all participants wanted to receive incidental results, regardless of the potential health implications. These results indicate that to maximize benefit, participant-level preferences should inform the decision about whether to return incidental findings.

Clinical genomic sequencing technologies are on the verge of allowing individualized care at reasonable cost.1 Patients and their families will soon receive information from clinical sequencing that has implications for clinical care, including information on consequences related to disease prognosis, treatment response or hereditary risk for disease.2 Clinical sequencing can also generate incidental findings, which are clinically relevant genetic variants for disorders unrelated to the reason for ordering the genetic testing. The decision of whether to provide information about incidental findings is complex because such results will have varying clinical validity (whether the genetic variant causes the disorder) and utility (whether effective medical treatment is available for the disorder).3,4 For example, although effective medical treatment may be available for some validated incidental findings, other incidental findings may not be validated as causing the disorder, and still others will be validated but not associated with effective treatment options.

To address in part the challenges surrounding the return of incidental findings, the American College of Medical Genetics and Genomics published recommendations for reporting incidental findings from clinical sequencing.5 The statement lists a minimum of 56 genes that laboratories should examine, with results reported to patients through the managing physician. This list includes genes with high-penetrance mutations (i.e., a high proportion of individuals with the mutation will exhibit clinical symptoms) validated to be associated with disorders for which medical interventions are available.

The original version of this statement did not “favour offering the patient a preference” for which results would be returned. The reasoning was that clinicians have a duty to prevent potential harm by telling patients about incidental findings. The working group that developed the recommendations further stated that it is impractical to provide the level of genetic counselling required for informed preference on all potential disorders.5 As such, the working group recommended that clinicians discuss with patients the possibility of receiving incidental findings from the list. It was argued that patient autonomy is preserved because patients can decline clinical sequencing if they prefer to not receive information about incidental findings.5 However, this rationale has been subject to debate because of its “all-or-none” nature, whereby patients must agree to receive information about incidental findings or clinical sequencing is not provided.6–9 In April 2014, in response to the ongoing debate, the statement was amended to include an “opt-out” option for patients who do not want to receive information about incidental findings.10

Notwithstanding the ethical debate, there is a lack of quantitative, preference-based economic evidence for the return of incidental genomic findings.8 It has been argued8 that this gap in evidence hindered development of the working group’s recommendation statement. More generally, evidence on preferences for the return of incidental findings is crucial for health policy, for health systems planning and for informing future lists that may include “many more genes.”8 We aimed to generate evidence on the personal utility that study participants from the Canadian public ascribe to the return of incidental genomic findings in the clinical setting. We chose participants from the general public because the public is the largest stakeholder in Canada’s publicly funded health care system.

## Methods

This study used a cross-sectional design in which a sample of the Canadian public 18 years of age or older stated their preferences for the return of incidental findings from genomic sequencing. To elicit preferences between alternatives and to estimate personal utility and willingness to pay, we used a discrete-choice experiment.11–16 In genomic medicine, personal utility is the worth that an individual ascribes to a genomic test.17 In economics, personal utility is quantified as a measure of well-being determined by eliciting individuals’ choices between different goods. It is assumed that individuals assign preferences to alternative goods and choose the most preferred good from the list of available alternatives; from these choices, indirect utility can be estimated.18

The discrete-choice method begins by identifying the key characteristics (called attributes) of a technology. Attributes are defined across a range of levels that affect the benefit that individuals will ascribe to a good. Experimental design techniques are then used to construct a series of choice tasks from combinations of the attribute levels (called scenario alternatives). Crucially, each task requires study participants to choose between 2 or more scenarios. The regression estimates are then interpreted as the utilities associated with each respective attribute level18 and are referred to as part-worth utilities. Total utility is estimated by summing over combinations of the part-worth utilities that describe a particular good. The inclusion of a cost attribute in the test description allows for a monetary estimate of utility, which is called willingness to pay.

### Questionnaire development and administration

Development of the questionnaire has been described previously.19 Briefly, we determined the attributes through literature review, consultation with experts and focus groups with patients undergoing genetic testing. We refined the questionnaire through pretest interviews19 with patients who underwent conventional genetic testing for familial colorectal cancer or polyposis syndromes. We selected the levels of each attribute according to their ability to accommodate a range of estimates that might be realized when incidental findings are returned. Each task included a choice between 2 alternatives (see Figure 1 for an example). A “no information” option was included to accommodate individuals who did not want to receive any information about incidental results.

![Figure 1:](http://www.cmaj.ca/https://www.cmaj.ca/content/cmaj/187/6/E190/F1.medium.gif)

[Figure 1:](http://www.cmaj.ca/content/187/6/E190/F1)

Figure 1: 
Example of a choice task question offered to participants. Each participant completed 16 such tasks. The scenarios differed on 5 attributes: threshold of penetrance for which information on incidental findings would be returned (i.e., disease risk), availability of an intervention for the identified disease, severity of health consequences of the disease, carrier status and cost of genetic testing. In each task, participants were asked to choose between 2 scenarios for receiving information about incidental findings; a third, opt-out alternative allowed for the possibility that participants would not want to receive any information about incidental findings.

We conducted a pilot study using a pan-Canadian sample of the public (*n* = 100). The final questionnaire included 5 attributes for each choice task: penetrance of the disease (or diseases) occurring at some point in the future (5% lifetime risk or higher, 40% lifetime risk or higher, 80% lifetime risk or higher, 90% lifetime risk or higher), treatability of the disease (no effective medical treatment or lifestyle change, recommended effective lifestyle change only, recommended effective medical treatment only, recommended effective medical treatment and lifestyle change), severity of the disease (mild, moderate, severe or very severe health consequences), carrier status (yes, no) and cost of receiving the results ($425, $750, $1000, $1500). The questionnaire was available in both French and English. The choice-based experimental design used D-optimal procedures to maximize the statistical efficiency of the model.20,21 This approach resulted in 80 choice tasks. Each participant was randomly assigned to 1 of 5 blocks containing 16 choice tasks.

A professional survey organization administered the questionnaire. The organization recruited participants using opportunistic sampling (e.g., through website pop-up messages) and through an Internet sampling panel of 295 983 Canadians. Participants who met the criteria for age (≥ 18 years) and region of residence according to Canadian population characteristics were eligible. Those who had participated in preference-based research in the previous 6 months were excluded. Those who agreed to participate were referred to a password-protected website. The questionnaire contained an introduction asking participants to imagine that they had been diagnosed with a serious disease and would undergo genetic testing. The introduction also provided an education module outlining the concept of incidental findings and the attributes and levels of the choice tasks. The University of British Columbia Behavioural Research Ethics Board, BC Cancer Agency, approved the study.

### Statistical analyses

We analyzed responses in Stata 12 software (StataCorp LP) using an error-component mixed logit model.22 This model allows for realistic estimates of the number of participants wanting information on incidental findings under various scenarios.22 It can also accommodate preference heterogeneity by allowing estimation of individual-specific utility values for receipt of results from incidental findings. The modelling approach requires specification of a distribution for each attribute level that is hypothesized to vary across participants. Parameters characterizing the mean and standard deviation of the heterogeneity distribution are estimated using maximum simulated likelihood techniques.11,22 The estimated mean and standard deviation parameters characterize the distribution of individual-level utility in the sampled population.

We used effects coding for each categorical attribute. We included cost as a continuous variable.14 We specified that attributes representing the threshold level for lifetime risk of disease, the treatability and severity of incidentally identified diseases, and information on carrier status would follow normal distributions. This allowed participants to have heterogeneous values for these attributes relative to the reference level. For example, for the 5% lifetime risk or higher attribute level, the normal distribution allowed participants to have positive or negative preferences about receiving this information. We specified the remaining attribute, cost, as fixed, to facilitate the statistical analysis.11

We examined the predicted uptake and willingness to pay for several scenarios related to the working group’s statement (as depicted in Figure 2). Predicted uptake was based on an estimate of the percentage of the population predicted to choose a particular scenario.13 We used the delta method to generate 95% confidence intervals (CIs) for these estimates.

![Figure 2:](http://www.cmaj.ca/https://www.cmaj.ca/content/cmaj/187/6/E190/F2.medium.gif)

[Figure 2:](http://www.cmaj.ca/content/187/6/E190/F2)

Figure 2: 
New policy scenarios examined to inform willingness to pay and to predict the percentage of the population that would find a defined scenario acceptable. The prevailing policies are based on published recommendations from the American College of Medical Genetics and Genomics.5

## Results

In total, 1200 of the 1965 individuals who responded to the email invitation completed all 16 choice questions (cooperation rate of 61%). Table 1 presents an overview of participants’ sociodemographic characteristics, relative to the Canadian adult population.

View this table:
[Table 1:](http://www.cmaj.ca/content/187/6/E190/T1)

Table 1: 
Participant characteristics compared with Canadian population

Participants expressed positive utility for receipt of incidental findings about diseases with a lifetime risk of at least 80% or at least 90%, with recommended effective medical or lifestyle interventions available, and with severe health outcomes, as well as for receipt of information about carrier status (Table 2). Disutility, or a reduction in well-being, was observed for incidental findings associated with disorders with no medical or lifestyle intervention available. Participants also expressed disutility for receiving information about disorders with mild health consequences and for a category in which incidental findings for all diseases with lifetime risk of 5% or higher were returned.

View this table:
[Table 2:](http://www.cmaj.ca/content/187/6/E190/T2)

Table 2: 
Regression estimates for part-worth utility*

We observed statistically significant preference heterogeneity for the utility that participants placed on several types of incidental findings. For example, all else being equal, we predicted that 16% of the participants would have a negative value for receipt of information about incidental findings for disorders with recommended effective medical treatment and lifestyle changes (Table 2). For carrier status, the model predicted that 42% of the sample would have a negative value for receiving this type of information.

Estimated willingness to pay and predicted uptake for alternative policy options are summarized in Table 3. Scenario 1 aligns with the working group’s recommendations5 (disorders with recommended medical treatment, 80% penetrance). The average willingness to pay for this scenario was $445 (95% CI $322–$567), and 66% of the participant population was predicted to find this scenario acceptable. The willingness to pay for scenario 2, the return of findings for which medical and lifestyle interventions are recommended, was $641 (95% CI $520–$762), with 73% of the population predicted to find the policy option to be acceptable.

View this table:
[Table 3:](http://www.cmaj.ca/content/187/6/E190/T3)

Table 3: 
Willingness to pay and predicted uptake for scenarios related to return of incidental findings

Scenario 3 examined the incremental willingness to pay of offering participants a choice between receipt of information about disorders for which only medical treatment is available or receipt of information about all disorders, regardless of treatment availability, compared with receipt of information only for disorders with recommended effective medical treatment available, as recommended by the American College of Medical Genetics and Genomics. The incremental willingness to pay for this scenario was $280 (95% CI $248–$313), which indicates a net positive value for access to the extended menu of options. The predicted probability of participants choosing an alternative in which information would be provided about both medically treatable and nontreatable disorders was 27%, whereas 49% of participants were predicted to choose the option of receiving incidental results only for medically treatable conditions. The estimated average willingness to pay for return of information on disorders with medical treatment available or information on all disorders, regardless of treatment availability, compared with returning no information was $725 (95% CI $600–$850), with a predicted uptake of 76%.

In a post hoc analysis, the sample was reweighted to better reflect the demographic characteristics of the Canadian population. We observed no differences in willingness to pay or in predicted uptake (details available from the authors).

## Interpretation

This article presents evidence on the cost–benefit trade-offs and personal utility that people ascribe to the receipt of information about incidental findings from clinical genomic sequencing. These results indicate that to maximize benefit, individuals’ preferences for disclosure should be considered and evaluated on an individual-level basis within a shared decision-making framework. Such an endeavour will require research into the best way to communicate the range of possible incidental findings.23

In relation to the statement of the American College of Medical Genetics and Genomics working group,5 we found support for returning information about high-penetrance incidental findings with clinical utility, if such information is preferred by the individual. Low-penetrance disorders generated disutility. These results indicate that there is a threshold of predicted disease penetrance below which people no longer want to receive information about incidental results. We also found evidence of benefit related to offering participants a choice between receipt of incidental findings for both treatable and nontreatable diseases or receipt of information about incidental findings for diseases with only medical interventions available.

Several studies have highlighted important qualitative factors when considering the return of incidental findings.19,24–29 The researchers have generally concluded that patient choice, health consequences, the risk of getting the disease, cost, family consequences and clinical validity are important considerations.19,25–30 A recent US conjoint analysis examined the return of incidental findings in the clinical research context and found support for returning clinically valid results for diseases that can be altered by medical and lifestyle interventions.30 However, that study did not examine willingness to pay or uptake of scenarios related to the statement of the working group.5 Furthermore, it did not explicitly examine trade-offs for returning results that did and did not have medical treatment available.

### Limitations

Our study had a number of limitations. To determine utility, we engaged a professional survey organization to recruit a pan-Canadian sample of the population. Although great care was taken in recruiting the participants, unobservable characteristics concerning participation in research of this type may introduce biases. We found statistically significant differences in demographic characteristics between the study participants and the Canadian population. Participants had higher education than the Canadian population; they also tended to be in the middle income ranges ($30 000 to $59 999). However, in our post hoc analysis in which the sample was reweighted to more closely represent the Canadian population, we observed no differences from our primary analysis. A second limitation related to the hypothetical nature of the choice task. We constructed an educational component for the discrete-choice survey similar to what could be presented in the clinic setting. However, because participants in this study would not be bound by their choices, the discrete-choice estimates may exhibit “hypothetical bias” (i.e., a difference between what people say they will choose and what they actually choose). In health economics, the criterion validity of discrete-choice experiments has been previously demonstrated,31 but research has shown that predicted probabilities of choice are accurate in the aggregate but not at the individual level.32

### Conclusion

In Canada, health care policy is the jurisdiction of provincial and territorial ministries. Although use of economic evidence is not mandated, cost-effectiveness and cost–benefit analyses are recommended for informing policy.33 Cost–benefit analyses in particular are recommended when evaluating the personal utility of genomic technologies.12,34 There can exist a degree of discomfort in assigning an estimate to personal utility.17 However, utility is essential for informing resource allocation decisions,35 because a policy alternative is economically more efficient if the utility from a program outweighs its cost. This situation is different from the nonmalfeasance principle in medicine, which emphasizes avoiding causing needless harm or injury according to reasonable standards.

Our results can be used to inform policy in the following ways. First, we estimated the probability of uptake related to several policy recommendations, which can be useful to predict demand for the return of incidental findings and as an input to economic decision-modelling. Second, the willingness-to-pay estimates can be used to inform reimbursement decisions. Third, the willingness-to-pay estimates can be used in cost–benefit analyses to examine the allocation of health resources.12 Finally, we found a diversity of interest about which incidental findings to return, if any. This result supports offering individual patients an informed choice about the types of incidental findings they prefer to receive.

## Footnotes

*   **Competing interests:** None declared.

*   This article has been peer reviewed.

*   **Contributors:** All authors participated in study conception and design and in drafting the manuscript and revising it for important intellectual content before final approval. Dean Regier and Reka Pataky were responsible for the statistical analysis. All authors agree to act as guarantors of the work.

*   **Funding:** This study was supported by funding from the Canadian Centre for Applied Research in Cancer Control, which receives core funding from the Canadian Cancer Society. Dean Regier, Gail Jarvik and David Veenstra receive support from the US National Human Genome Research Institute (grant U01 HG0006507-01).

*   Received March 9, 2015.
*   Accepted June 15, 2015.

## References

1.  Manolio TA, Chisholm RL, Ozenberger B, et al. Implementing genomic medicine in the clinic: the future is here. Genet Med 2013; 15:258–67.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1038/gim.2012.157&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=23306799&link_type=MED&atom=%2Fcmaj%2F187%2F6%2FE190.atom) 

2.  Biesecker LG. Opportunities and challenges for the integration of massively parallel genomic sequencing into clinical practice: lessons from the ClinSeq project. Genet Med 2012;14: 393–8.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1038/gim.2011.78&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=22344227&link_type=MED&atom=%2Fcmaj%2F187%2F6%2FE190.atom) 

3.  Knoppers BM, Joly Y, Simard J, et al. The emergence of an ethical duty to disclose genetic research results: international perspectives. Eur J Hum Genet 2006;14:1170–8.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1038/sj.ejhg.5201690&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=16868560&link_type=MED&atom=%2Fcmaj%2F187%2F6%2FE190.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000241577900006&link_type=ISI) 

4.  Biesecker LG. Incidental variants are critical for genomics. Am J Hum Genet 2013;92:648–51.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/j.ajhg.2013.04.001&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=23643378&link_type=MED&atom=%2Fcmaj%2F187%2F6%2FE190.atom) 

5.  Green RC, Berg JS, Grody WW, et al. ACMG recommendations for reporting of incidental findings in clinical exome and genome sequencing. Genet Med 2013;15:565–74.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1038/gim.2013.73&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=23788249&link_type=MED&atom=%2Fcmaj%2F187%2F6%2FE190.atom) 

6.  Bombard Y, Robson M, Offit K. Revealing the incidentalome when targeting the tumor genome. JAMA 2013;310:795–6.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1001/jama.2013.276573&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=23982363&link_type=MED&atom=%2Fcmaj%2F187%2F6%2FE190.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000323561400015&link_type=ISI) 

7.  Klitzman R, Appelbaum PS, Chung W. Return of secondary genomic findings vs patient autonomy: implications for medical care. JAMA 2013;310:369–70.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1001/jama.2013.41709&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=23917282&link_type=MED&atom=%2Fcmaj%2F187%2F6%2FE190.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000322121500009&link_type=ISI) 

8.  Ross LF, Rothstein MA, Clayton EW. Mandatory extended searches in all genome sequencing: “incidental findings,” patient autonomy, and shared decision making. JAMA 2013; 310:367–8.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1001/jama.2013.41700&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=23917281&link_type=MED&atom=%2Fcmaj%2F187%2F6%2FE190.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000322121500008&link_type=ISI) 

9.  Wolf SM, Annas GJ, Elias S. Point-counterpoint. Patient autonomy and incidental findings in clinical genomics. Science 2013;340:1049–50.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6Mzoic2NpIjtzOjU6InJlc2lkIjtzOjEzOiIzNDAvNjEzNi8xMDQ5IjtzOjQ6ImF0b20iO3M6MjE6Ii9jbWFqLzE4Ny82L0UxOTAuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

10. ACMG Board of Directors. ACMG policy statement: updated recommendation regarding analysis and reporting of secondary findings in clinical genome-scale sequencing. Genet Med 2014;17: 68–9.
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=25356965&link_type=MED&atom=%2Fcmaj%2F187%2F6%2FE190.atom) 

11. Regier DA, Ryan M, Phimister E, et al. Bayesian and classical estimation of mixed logit: an application to genetic testing. J Health Econ 2009;28:598–610.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/j.jhealeco.2008.11.003&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=19345433&link_type=MED&atom=%2Fcmaj%2F187%2F6%2FE190.atom) 

12. Regier DA, Friedman JM, Marra CA. Value for money? Array genomic hybridization for diagnostic testing for genetic causes of intellectual disability. Am J Hum Genet 2010;86:765–72.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/j.ajhg.2010.03.009&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=20398885&link_type=MED&atom=%2Fcmaj%2F187%2F6%2FE190.atom) 

13. Ryan M, Gerard K, Amaya-Amaya M. Using discrete choice experiments to value health and health care. Dordrecht (Netherlands): Springer; 2007.
    
    

14. Regier DA, Friedman JM, Makela N, et al. Valuing the benefit of diagnostic testing for genetic causes of idiopathic developmental disability: willingness to pay from families of affected children. Clin Genet 2009;75:514–21.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1111/j.1399-0004.2009.01193.x&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=19508416&link_type=MED&atom=%2Fcmaj%2F187%2F6%2FE190.atom) 

15. Foster MW, Mulvihill JJ, Sharp RR. Evaluating the utility of personal genomic information. Genet Med 2009;11:570–4.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1097/GIM.0b013e3181a2743e&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=19478683&link_type=MED&atom=%2Fcmaj%2F187%2F6%2FE190.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000269273900003&link_type=ISI) 

16. Grosse SD, McBride CM, Evans JP, et al. Personal utility and genomic information: look before you leap. Genet Med 2009; 11:575–6.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1097/GIM.0b013e3181af0a80&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=19623080&link_type=MED&atom=%2Fcmaj%2F187%2F6%2FE190.atom) 

17. Feero WG, Wicklund C, Veenstra DL. The economics of genomic medicine: insights from the IOM Roundtable on Translating Genomic-Based Research for Health. JAMA 2013; 309:1235–6.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1001/jama.2013.113&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=23532238&link_type=MED&atom=%2Fcmaj%2F187%2F6%2FE190.atom) 

18. 1.  Zarembka P
    
    McFadden D. Conditional logit analysis of qualitative choice behavior. In: Zarembka P, editor. Frontiers in econometrics. New York (NY): Academic Press; 1974. p. 105–42.
    
    

19. Bennette CS, Trinidad SB, Fullerton SM, et al. Return of incidental findings in genomic medicine: measuring what patients value — development of an instrument to measure preferences for information from next-generation testing (IMPRINT). Genet Med 2013;15:873–81.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1038/gim.2013.63&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=23722871&link_type=MED&atom=%2Fcmaj%2F187%2F6%2FE190.atom) 

20. Kuhfeld WF, Tobias RD, Garratt M. Efficient experimental-design with marketing-research applications. J Mark Res 1994; 31:243–64.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.2307/3152197&link_type=DOI) 

21. Reed Johnson F, Lancsar E, Marshall D, et al. Constructing experimental designs for discrete-choice experiments: report of the ISPOR Conjoint Analysis Experimental Design Good Research Practices Task Force. Value Health 2013;16:3–13.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/j.jval.2012.08.2223&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=23337210&link_type=MED&atom=%2Fcmaj%2F187%2F6%2FE190.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000314060200002&link_type=ISI) 

22. Train K. Discrete choice methods with simulation. 2nd ed. Cambridge (UK): Cambridge University Press; 2009.
    
    

23. Goddard KA, Whitlock EP, Berg JS, et al. Description and pilot results from a novel method for evaluating return of incidental findings from next-generation sequencing technologies. Genet Med 2013;15:721–8.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1038/gim.2013.37&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=23558254&link_type=MED&atom=%2Fcmaj%2F187%2F6%2FE190.atom) 

24. Townsend A, Adam S, Birch PH, et al. “I want to know what’s in Pandora’s box”: comparing stakeholder perspectives on incidental findings in clinical whole genomic sequencing”. Am J Med Genet A 2012;158A:2519–25.
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=22903777&link_type=MED&atom=%2Fcmaj%2F187%2F6%2FE190.atom) 

25. Facio FM, Eidem H, Fisher T, et al. Intentions to receive individual results from whole-genome sequencing among participants in the ClinSeq study. Eur J Hum Genet 2013;21:261–5.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1038/ejhg.2012.179&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=22892536&link_type=MED&atom=%2Fcmaj%2F187%2F6%2FE190.atom) 

26. Bollinger JM, Scott J, Dvoskin R, et al. Public preferences regarding the return of individual genetic research results: findings from a qualitative focus group study. Genet Med 2012;14: 451–7.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1038/gim.2011.66&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=22402755&link_type=MED&atom=%2Fcmaj%2F187%2F6%2FE190.atom) 

27. Daack-Hirsch S, Driessnack M, Hanish A, et al. Information is information: a public perspective on incidental findings in clinical and research genome-based testing. Clin Genet 2013;84: 11–8.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1111/cge.12167&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=23590238&link_type=MED&atom=%2Fcmaj%2F187%2F6%2FE190.atom) 

28. Nicholls SG, Wilson BJ, Craigie SM, et al. Public attitudes towards genomic risk profiling as a component of routine population screening. Genome 2013;56:626–33.
    
    

29. Yu JH, Crouch J, Jamal SM, et al. Attitudes of non-African American focus group participants toward return of results from exome and whole genome sequencing. Am J Med Genet A 2014; 164A:2153–60.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1002/ajmg.a.36610&link_type=DOI) 

30. Murphy Bollinger J, Bridges JF, Mohamed A, et al. Public preferences for the return of research results in genetic research: a conjoint analysis. Genet Med 2014;16:932–9.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1038/gim.2014.50&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=24854226&link_type=MED&atom=%2Fcmaj%2F187%2F6%2FE190.atom) 

31. Telser H, Zweifel P. Measuring willingness-to-pay for risk reduction: an application of conjoint analysis. Health Econ 2002;11:129–39.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1002/hec.653&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=11921311&link_type=MED&atom=%2Fcmaj%2F187%2F6%2FE190.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000174474400004&link_type=ISI) 

32. Linley WG, Hughes DA. Decision-makers’ preferences for approving new medicines in wales: a discrete-choice experiment with assessment of external validity. Pharmacoeconomics 2013;31:345–55.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1007/s40273-013-0030-0&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=23516033&link_type=MED&atom=%2Fcmaj%2F187%2F6%2FE190.atom) 

33. Guidelines for the economic evaluation of health technologies: Canada. Ottawa: Canadian Agency for Drugs and Technologies in Health; 2006.
    
    

34. Buchanan J, Wordsworth S, Schuh A. Issues surrounding the health economic evaluation of genomic technologies. Pharmacogenomics 2013;14:1833–47.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.2217/pgs.13.183&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=24236483&link_type=MED&atom=%2Fcmaj%2F187%2F6%2FE190.atom) 

35. Drummond M, O’Brien B, Stoddart G, et al. Methods for the economic evaluation of health care programmes. 2nd ed. Oxford (UK): Oxford University Press; 1997.