Health care utilization and outcomes of patients seen by virtual urgent care versus in-person emergency department care
=======================================================================================================================

* Shelley L. McLeod
* Jean-Eric Tarride
* Shawn Mondoux
* J. Michael Paterson
* Lesley Plumptre
* Emily Borgundvaag
* Katie N. Dainty
* Joy McCarron
* Howard Ovens
* Justin N. Hall

## Abstract

**Background:** Virtual urgent care (VUC) is intended to support diversion of patients with low-acuity complaints and reduce the need for in-person emergency department visits. We aimed to describe subsequent health care utilization and outcomes of patients who used VUC compared with similar patients who had an in-person emergency department visit.

**Methods:** We used patient-level encounter data that were prospectively collected for patients using VUC services provided by 14 pilot programs in Ontario, Canada. We linked the data to provincial administrative databases to identify subsequent 30-day health care utilization and outcomes. We defined 2 subgroups of VUC users; those with a documented prompt referral to an emergency department by a VUC provider, and those without. We matched patients in each cohort to an equal number of patients presenting to an emergency department in person, based on encounter date, medical concern and the logit of a propensity score. For the subgroup of patients not promptly referred to an emergency department, we matched patients to those who were seen in an emergency department and then discharged home.

**Results:** Of the 19 595 patient VUC visits linked to administrative data, we matched 2129 patients promptly referred to the emergency department by a VUC provider to patients presenting to the emergency department in person. Index visit hospital admissions (9.4% v. 8.7%), 30-day emergency department visits (17.0% v. 17.5%), and hospital admissions (12.9% v. 11.0%) were similar between the groups. We matched 14 179 patients who were seen by a VUC provider with no documented referral to the emergency department. Patients seen by VUC were more likely to have a subsequent in-person emergency department visit within 72 hours (13.7% v. 7.0%), 7 days (16.5% v. 10.3%) and 30 days (21.9% v. 17.9%), but hospital admissions were similar within 72 hours (1.1% v. 1.3%), and higher within 30 days for patients who were discharged home from the emergency department (2.6% v. 3.4%).

**Interpretation:** The impact of the provincial VUC pilot program on subsequent health care utilization was limited. There is a need to better understand the inherent limitations of virtual care and ensure future virtual providers have timely access to in-person outpatient resources, to prevent subsequent emergency department visits.

[See related editorial at www.cmaj.ca/lookup/doi/10.1503/cmaj.231496](http://www.cmaj.ca/lookup/volpage/195/E1483)

The COVID-19 pandemic has had a substantial impact on the way people in Canada interact with health care systems.1,2 In the early stages of the pandemic, when physical distancing was strongly encouraged, it was difficult to arrange a nonurgent, in-person health care visit. In-person primary care visits declined by nearly 80%, and emergency department visits decreased by 50%.3–5 Although technologies to deliver health care through means other than face-to-face contact have been available for decades, the beginning of the COVID-19 pandemic saw large growth and rapid adoption of virtual care.6,7

As part of the COVID-19 pandemic response, in the fall of 2020 the Ontario Ministry of Health approved up to $4 million to fund a pilot program involving 14 virtual urgent care (VUC) initiatives across the province.8 This funding was intended to support emergency department diversion of patients with low-acuity complaints and reduce the need for in-person visits whenever possible. The design and implementation of the 14 emergency department–led virtual sites and the demographic characteristics and experience of patients using VUC have been described elsewhere;9,10 the 14 different sites had various start dates, operating hours, screening and staffing models. Each site posted a list of presenting complaints that would be suitable for VUC, and those that should be assessed in person. All patients accessing VUC were assessed by an emergency physician. More information describing the design of the pilot programs can be found in Appendix 1 (available at [www.cmaj.ca/lookup/doi/10.1503/cmaj.230492/tab-related-content](http://www.cmaj.ca/lookup/doi/10.1503/cmaj.230492/tab-related-content)).

Although the shift to virtual care was intended to maintain safe access to medical care during the pandemic, it occurred despite lack of evidence regarding the comparability of in-person and virtual care models. Given that virtual care options are now more widespread across the health care system, it is important to understand how virtual care compares with traditional in-person care in the emergency department setting. We sought to describe the characteristics and subsequent health care utilization and outcomes of patients using VUC services compared with a matched cohort of similar patients who received traditional in-person emergency department care.

## Methods

### Study design and setting

We used patient-level encounter data that were prospectively collected for all patients using VUC services provided by 14 emergency department–led pilot sites (including 3 pediatric sites), in Ontario, Canada, from December 2020 to September 2021. The characteristics of the pilot programs are described in Appendix 1. The 14 sites approved for VUC pilot funding represented a mix of pediatric, northern, urban, academic and community sites across the province.9,10

### Data sources

As part of the funding agreement, each participating VUC site was required to collect and report a minimum of 6 months of standardized patient-level encounter data, which were sent electronically via secure file transfer to Ontario Health. Standardized patient-level encounter data included the patient’s health insurance number, age, sex, primary care physician (PCP), date and time of the VUC visit, presenting complaint, acuity level (Canadian Triage and Acuity Scale score 1–5), and the VUC discharge diagnosis (physician diagnosis) and disposition (discharged home, referred to emergency department, referred to PCP, referred to specialist, left without being seen, referred to COVID-19 or outpatient clinic, other). For every patient encounter, the VUC provider documented all patient data at the time of the VUC visit.

We used patient-level encounter data from each VUC visit to link to province-wide administrative health care databases held at ICES ([www.ices.on.ca](http://www.ices.on.ca)) to identify subsequent health care utilization (emergency department visits, hospital admissions, outpatient specialist or PCP visits) within 30 days. Outpatient specialist or PCP visits were identified based on provincial billing data and location code. We obtained information regarding emergency department visits from the Canadian Institute for Health Information (CIHI) National Ambulatory Care Reporting System, which contains abstracted data on all emergency department visits in Ontario. The CIHI Discharge Abstract Database contains information on all acute care hospital admissions and in-patient surgical procedures in the province. The Ontario Health Insurance Plan (OHIP) Claims History Database contains all physician billings for medically necessary care. The OHIP Registered Persons Database contains demographic and place of residence information, health insurance status and vital status for all people in Ontario, including out-of-hospital deaths. Patients were tracked in these databases at the person level using unique, encoded identifiers. Ontario has publicly funded health care coverage for medically necessary care; therefore, these databases contain information on most of the health care utilization in the province.

### Study participants

We required a valid linkable OHIP number for inclusion, and individuals had to have been eligible for OHIP coverage for a minimum of 365 days before the index date. We excluded VUC records if they contained invalid (e.g., incorrect OHIP number) or incomplete data (e.g., patients registered but did not show up for their VUC appointment); we also excluded virtual visits by non-Ontario residents.

As we expected differences in the characteristics of people using VUC or in-person emergency department care, we defined 2 subgroups of VUC users: those attending a VUC appointment, who were promptly referred to an in-person emergency department; and those seen by a VUC provider, with no referral to an in-person emergency department. We matched patients with a VUC discharge disposition of “referred to the emergency department” to a cohort of patients who presented in person to the emergency department with any disposition, and we matched patients seen by a VUC provider with no documented referral to an in-person emergency department to a cohort of patients who presented in person to the emergency department and were discharged home. This allowed us to assess subsequent health care utilization (emergency department visits, hospital admissions, and specialist and primary care visits) and death in patients with similar baseline characteristics. We included patients missing a VUC discharge disposition in the cohort with no documented referral to the emergency department. For each patient in the VUC cohort, we 1:1 greedy-matched a patient presenting to an emergency department in person on the basis of encounter date (± 14 d), presenting Canadian Emergency Department Information System complaint and the logit of a propensity score computed from each patient’s age, sex, Statistics Canada Census neighbourhood income quintile, urban or rural residence, Ontario Marginalization Index quintiles related to ethnic diversity, residential instability, material deprivation and dependency, whether patients were rostered with a PCP, number of major Aggregated Diagnostic Groups (ADG) derived from the Johns Hopkins ACG System (version 10; grouped as 0–4, 5–9, 10–14, ≥ 15), selected chronic diseases identified using ICES administrative data case definitions (e.g., asthma, congestive heart failure, hypertension, chronic obstructive pulmonary disease, diabetes and dementia), the time and day of the index visit, number of PCP and specialist physician visits, emergency department visits and hospital admissions in the year preceding the index date. We used a caliper width of 0.2 of the standard deviation of the logit of the propensity score. We evaluated balance in baseline covariates for each cohort using standardized differences, with values less than 0.10 indicating that the groups were well matched.11

### Statistical analysis

We summarized data using means with standard deviations or frequencies, where appropriate. We compared proportional differences using χ2 statistics. All ICES-based analyses were performed using linked, coded data in SAS (version 9.4, SAS Institute). We conducted subsequent statistical analyses using aggregate data in SPSS (version 28.0, IBM Corporation).

### Ethics approval

The study received ethics approval from the Research Ethics Board at Sinai Health (21-003-E).

## Results

Of the 22 278 patient encounters captured in the provincial VUC program, we were able to match 19 595 (88.0%) to administrative health care databases (Appendix 2, available at available at [www.cmaj.ca/lookup/doi/10.1503/cmaj.230492/tab-related-content](http://www.cmaj.ca/lookup/doi/10.1503/cmaj.230492/tab-related-content)). The baseline demographic characteristics for all eligible VUC encounters, classified by adult and pediatric VUC pilot sites, are shown in Table 1. Mean patient age was 28 years, 60% were female and 85% had a PCP. Most virtual visits were for low-acuity complaints and 70% were managed by the VUC provider without the need for emergent in-person emergency department assessment or referral elsewhere.

View this table:
[Table 1:](http://www.cmaj.ca/content/195/43/E1463/T1)

Table 1: 
Baseline demographic characteristics for all eligible virtual urgent care (VUC) records, classified by adult and pediatric VUC pilot sites*

Of the 19 595 patient encounters linked to administrative data, 12.5% had an in-person emergency department visit within 72 hours and 21.5% had an in-person emergency department visit within 30 days of the index VUC visit; 2.1% had a hospital admission within 72 hours and 3.8% were admitted within 30 days of the index VUC visit (Table 2). There were few (< 0.03%, number suppressed to protect patient privacy) deaths within 30 days, none from the pediatric sites.

View this table:
[Table 2:](http://www.cmaj.ca/content/195/43/E1463/T2)

Table 2: 
Health care utilization for all eligible virtual urgent care (VUC) records, classified by adult and pediatric VUC pilot sites*

Of the 17 034 patients with a known VUC disposition, 2931 (17.2%) were referred to the emergency department (Table 1). Of those, 669 (22.8%) did not present to the emergency department within 72 hours of their VUC visit, but their 30-day emergency department visits were lower (14.3% v. 21.5%) and hospital admissions were similar (3.3% v. 3.8%) than those of the overall cohort (Appendix 3, available at [www.cmaj.ca/lookup/doi/10.1503/cmaj.230492/tab-related-content](http://www.cmaj.ca/lookup/doi/10.1503/cmaj.230492/tab-related-content), and Table 2). Of the 2262 encounters with patients who were promptly referred to the emergency department by a VUC provider and presented to the emergency department within 72 hours, records from 2150 patients were available to be matched to provincial administrative databases. We matched 2129 (99.0%) to similar patients who presented to the emergency department in person. Results of the matched baseline characteristics are provided in Appendix 4A (available at [www.cmaj.ca/lookup/doi/10.1503/cmaj.230492/tab-related-content](http://www.cmaj.ca/lookup/doi/10.1503/cmaj.230492/tab-related-content)). Baseline characteristics were well matched between groups. Although patients presenting to the emergency department in person were more likely to arrive by ambulance, index visit admissions, 30-day emergency department visits and hospital admissions were similar between groups (Table 3). Mean hospital length of stay was higher for patients who were referred to the emergency department by a VUC provider (7.5 d v. 5.1 d; Δ 2.4 d, 95% confidence interval [CI] 1.6 d to 3.2 d). Patients who initially sought VUC were more likely to have another VUC visit within 72 hours, 7 days and 30 days than patients who presented to the emergency department. They were also more likely to have a subsequent specialist visit within 7 days (24.0% v. 17.5%; Δ 6.5%, 95% CI 4.1% to 9.0%) and 30 days (48.6% v. 37.3%; Δ 11.3%, 95% CI 8.4% to 14.3%).

View this table:
[Table 3:](http://www.cmaj.ca/content/195/43/E1463/T3)

Table 3: 
Emergency department (ED) characteristics and health care utilization for virtual urgent care prompt referrals who presented to ED within 72 hours versus matched in-person controls

Of the 16 664 patient encounters who were seen by a VUC provider with no documented emergency department referral, records from 14 498 patients were available to be matched to provincial administrative databases (Appendix 2). We matched 14 179 (97.8%) to similar patients who presented to the emergency department in person and were discharged home. Results of the matched baseline characteristics are provided in Appendix 4B. Baseline characteristics were well matched between groups. Patients using VUC services were more likely to have an in-person emergency department visit within 72 hours (13.7% v. 7.0%; Δ 6.7%, 95% CI 6.1% to 7.5%), 7 days (16.5% v. 10.3%; Δ 6.2%, 95% CI 5.4% to 7.0%) and 30 days (21.9% v. 17.9%; Δ 4.0%, 95% CI 3.2% to 5.0%), but the percentage admitted to hospital within 72 hours (1.1% v. 1.3%; Δ −0.2%, 95% CI −0.5% to 0.0%) and 7 days (1.6% v. 1.9%; Δ −0.3%, 95% CI −0.6% to 0.0%) was similar to those presenting to the emergency department in person (Table 4). However, the percentage admitted to hospital within 30 days was greater for patients who visited the emergency department in person (3.4% v. 2.6%; Δ 0.8%, 95% CI 0.4% to 1.2%). Mean hospital length of stay was higher for patients initially presenting to VUC (6.2 d v. 5.2 d; Δ 1.0 d, 95% CI 0.7 d to 1.2 d). Few deaths (1–5) occurred in the VUC group, and no deaths occurred among matched patients. Deaths were not significantly different between groups.

View this table:
[Table 4:](http://www.cmaj.ca/content/195/43/E1463/T4)

Table 4: 
Health care utilization for patients seen by a virtual urgent care provider with no emergency department (ED) referral plus matched in-person ED controls who were discharged from the ED

The most common presenting complaints for patients who had a subsequent emergency department visit within 72 hours (Table 5) and 30 days (Table 6) after a VUC visit were fever and abdominal pain, with COVID-19 being the top known emergency department discharge diagnosis. The most common presenting complaints for patients who had a subsequent emergency department visit within 72 hours after an in-person emergency department visit were imaging tests and abnormal laboratory values and abdominal pain, with unspecified abdominal pain being the top known emergency department discharge diagnosis (Table 5).

View this table:
[Table 5:](http://www.cmaj.ca/content/195/43/E1463/T5)

Table 5: 
Characteristics of patients who visited the emergency department (ED) within 72 hours after being seen by virtual urgent care without an ED referral compared with patients discharged home after an in-person ED visit

View this table:
[Table 6:](http://www.cmaj.ca/content/195/43/E1463/T6)

Table 6: 
Characteristics of patients who visited the emergency department (ED) within 30 days after being seen by virtual urgent care without an ED referral compared with patients discharged home after an in-person ED visit

## Interpretation

We found that the overall impact of the provincial VUC pilot program on subsequent health care utilization was not significant. Patients referred promptly to the emergency department by a VUC provider had rates of health care utilization similar to those of patients who presented in person to the emergency department. Patients seen by a VUC provider with no further referral were more likely to have an in-person emergency department visit within 72 hours, 7 days and 30 days, and mean length of hospital admission was longer than for patients who presented to the emergency department in person and were discharged home. The presenting complaints for patients with subsequent emergency department visits appeared to be different between groups. Patients who presented to the emergency department in person and were discharged home returned more for diagnostic imaging and repeat blood work, while VUC patients may have had symptoms more likely to require an in-person assessment.

Virtual care may be an appropriate alternative health care option for patients who have non–life-threatening medical concerns such as minor injuries, coughs, colds and other illnesses that align with a list of best-practice conditions for virtual care.12–17 It may also be a convenient option for patients who have difficulty accessing in-person health care, such as those who live in rural or remote areas or have mobility issues, those with no PCP or those who cannot access their PCP in a timely fashion.18–20 Virtual care may also be appropriate for patients seeking guidance on whether an in-person emergency department visit is needed, but given the low acuity of presenting complaints, perhaps nurse practitioners, physician assistants or PCPs — as opposed to emergency physicians — may be better suited to provide these virtual services as part of a “primary care first” strategy, with the opportunity to escalate to a VUC emergency department physician for advice before recommending an in-person emergency department visit. This is especially true given the current crisis in emergency department staffing and severe emergency department workforce shortages.21

The services offered by VUC clinicians are inherently different from the in-person assessments their emergency department colleagues can provide, which may partially explain the subsequent greater health care utilization by patients seen by a VUC provider in this study. Virtual urgent care providers are not able to provide a comprehensive physical examination and may not have access to real-time laboratory tests, imaging and the patient’s previous medical records, which may limit their ability to accurately diagnose certain conditions.22–24 Future directions in the provision of virtual care should ensure that VUC providers have timely access (within 24 h) to outpatient laboratories and imaging, and the ability to connect with a practitioner in person, ideally with the patient’s longitudinal PCP, especially for higher-acuity or higher risk–presenting complaints, or if an in-person physical examination is required. Timely access to in-person outpatient resources may help decrease subsequent health care utilization and reduce duplication at the system, regional and hospital levels.

Similar to previous studies,25–28 we found that patients accessing VUC services tended to be middle-aged, female, well-educated, predominantly English speaking, urban residents of high socioeconomic status, suggesting that there may be inequitable awareness of and access to VUC services. Users of VUC were younger than those accessing in-person emergency department care. Mean patient age for the adult VUC sites was 41 years, which is younger than the provincial average of 47 years.29 We found that patients seen by a VUC provider and discharged home were more likely to have an in-person emergency department visit within 72 hours, 7 days and 30 days than those who presented directly to the emergency department. Lapointe-Shaw and colleagues found that virtual walk-in patients were less likely to have a follow-up in-person visit with the same physician, more likely to have another virtual visit, and twice as likely to visit the emergency department within 30 days of the initial virtual appointment.26 In a retrospective study of more than 16 000 unique emergency department encounters, Shah and colleagues found that virtual follow-up visits after an emergency department discharge were associated with an increased rate of return emergency department visits and hospital admission within 30 days compared with in-person follow-up visits, even after adjusting for sociodemographic factors, acuity of illness and medical complexity.24 The authors suggested that the increased health care utilization after virtual follow-up visits may be a result of the inherent limitation in the ability of virtual clinicians to physically examine patients, which may compel clinicians to have a lower threshold for referring patients back to the emergency department for an in-person evaluation if they have any ongoing symptoms.

Future research should attempt to identify quality-of-care indicators and best practices of virtual care as part of building a culture of continuous improvement. Similar to the Emergency Department Return Visit Quality Program,30,31 future work should identify, audit and investigate underlying causes of subsequent health care utilization after a VUC visit and determine what needs to be addressed and changed, moving forward, and learn from the successes and challenges of the initial pilot program. We previously asked each VUC program lead to describe their local pilot initiative, share facilitators and barriers to adoption of VUC services, and summarize lessons learned for future VUC design and development.9 Common facilitators included local champions to guide VUC program delivery, provincial funding support, incorporating patients throughout the planning process, and multimodal marketing and promotions. Common barriers included behaviour change strategies to support adoption of a new service; access to high-quality information technology to support new workflow models that consider privacy, risk and legal perspectives; and standardized data collection to support overall objective impact assessments. Although the heterogeneity of program implementation respected local autonomy, it also presented challenges for sustainability efforts and future funding considerations. Future research should engage community members from vulnerable populations to determine strategies to improve awareness and uptake of virtual care among equity-deserving and underserved populations, especially in rural and remote communities where access to health care is more challenging. Researchers may also wish to examine what data should be collected to help inform VUC models, moving forward, to achieve the correct balance between virtual and in-person care that optimizes overall access, quality and value for money.

### Limitations

Patients who use VUC services seem to be inherently different from patients who present in person to the emergency department. We decided to split the cohort of patients using VUC services into those promptly referred to the emergency department for in-person care and those who saw a VUC provider with no documented referral to the emergency department. We included 2561 VUC encounters that had a missing discharge disposition in our “No emergency department referral” cohort, but it is possible that some of these patients may have been referred to the emergency department by the VUC provider during their initial VUC visit. Therefore, we may have underestimated the differences in 72-hour, 7-day and 30-day emergency department visits between groups. Although we matched on many important baseline characteristics, there may be some variables (e.g., acuity score) not included in our propensity score match that could be related both to a decision to use VUC and to outcomes, so the risk of residual confounding remains. The generalizability of our results to populations with different demographics is uncertain. In addition, patients using VUC services in 1 geographic location may share a similar health care pathway and may be more alike than patients in other locations and different care pathways. We documented the physician visits associated with the index day, but we did not have access to the exact time of the VUC or in-person emergency department visit and, as such, temporality is unknown between the primary care visit, specialist visit and VUC visit. The physician categories in OHIP billing data are not always clearly defined or standardized, which may lead to inconsistent categorization across different health care providers or even the same provider. Some VUC providers may have been misclassified as a PCP or as a specialist (e.g., pediatrician). We also do not know if the VUC users would have attended an emergency department in the absence of VUC. They may have waited to access their PCP, a walk-in clinic or other health care service. Future work should compare the health care expenditures of VUC to these alternative services. It is not clear if there was a difference in health care utilization dependent on mode of VUC delivery (video v. phone) or initial screening process (self-screening v. triage nurse). Finally, the VUC patient encounters included in this study were from a 10-month period early in the COVID-19 pandemic. Therefore, our findings may not be generalizable to VUC services or health care utilization outside the study period.

## Conclusion

We found no overall impact of the provincial VUC pilot program on both subsequent emergency department visits and hospital admissions, although an important percentage of VUC patients subsequently attended an emergency department in person. These findings highlight the need to better understand the inherent limitations of virtual care and ensure future VUC providers have timely access to in-person outpatient resources for follow-up, to reduce subsequent emergency department visits and ensure appropriate use of emergency department services.

## Footnotes

*   **Competing interests:** Joy McCarron is a paid employee of Ontario Health. Shawn Mondoux reports research funding from the Juravinski Research Institute and PSI Foundation, as well as travel support from Mitacs. Howard Ovens is a paid advisor to the Ministry of Health and in that capacity has provided executive sponsorship for the virtual urgent care evaluation. No oher competing interests were declared.

*   This article has been peer reviewed.

*   **Contributors:** Shelley McLeod, Jean-Eric Tarride, Shawn Mondoux, J. Michael Paterson, Lesley Plumptre, Emily Borgundvaag and Justin Hall contributed to the conception and design of the work. J. Michael Paterson, Lesley Plumptre and Emily Borgundvaag contributed to the acquisition and analysis of the data. All of the authors contributed to the interpretation of data. Shelley McLeod drafted the manuscript. All of the authors revised it critically for important intellectual content, gave final approval of the version to be published and agreed to be accountable for all aspects of the work.

*   **Funding:** Ontario Health provided operational funding to each of the virtual urgent care pilot sites. The funding body had no role in the design, analysis, interpretation, writing or approval of this article. The views expressed in this article are the views of the authors and do not necessarily reflect the views of the province.

*   **Data sharing:** The data set from this study is held securely in coded form at ICES. Although legal data sharing agreements between ICES and data providers (e.g., health care organizations and governments) prohibit ICES from making the data set publicly available, access may be granted to those who meet prespecified criteria for confidential access, available at [https://www.ices.on.ca/DAS](https://www.ices.on.ca/DAS) (das{at}ices.on.ca). The full data set creation plan and underlying analytical code are available from the authors upon request, understanding that the computer programs may rely upon coding templates or macros that are unique to ICES and are therefore either inaccessible or may require modification.

*   **Disclaimer:** This study was supported by ICES, which is funded by an annual grant from the Ontario Ministry of Health (MOH) and the Ministry of Long-Term Care (MLTC). This document used data adapted from the Statistics Canada Postal CodeOM Conversion File, which is based on data licensed from Canada Post Corporation, and/or data adapted from the Ontario Ministry of Health Postal Code Conversion File, which contains data copied under license from Canada Post Corporation and Statistics Canada. Parts of this material are based on data and information compiled and provided by Canadian Institute for Health Information and the Ontario MOH. The analyses, conclusions, opinions and statements expressed herein are solely those of the authors and do not reflect those of the funding or data sources; no endorsement is intended or should be inferred. The authors thank the Toronto Community Health Profiles Partnership for providing access to the Ontario Marginalization Index. Ontario Health provided operational funding to each of the virtual urgent care pilot sites. The funding body had no role in the design, analysis, interpretation, writing or approval of this article. The views expressed in this article are the views of the authors and do not necessarily reflect the views of the province.

*   Accepted September 21, 2023.

This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY-NC-ND 4.0) licence, which permits use, distribution and reproduction in any medium, provided that the original publication is properly cited, the use is noncommercial (i.e., research or educational use), and no modifications or adaptations are made. See: [https://creativecommons.org/licenses/by-nc-nd/4.0/](https://creativecommons.org/licenses/by-nc-nd/4.0/)

## References

1.  Health workforce in Canada: in focus (including nurses and physicians). Ottawa: Canadian Institute for Health Information; updated 2022 Nov. 17. Available: [https://www.cihi.ca/en/health-workforce-in-canada-in-focus-including-nurses-and-physicians](https://www.cihi.ca/en/health-workforce-in-canada-in-focus-including-nurses-and-physicians) (accessed 2023 Apr. 10).
    
    

2.  Canadian Digital Health Survey 2021: what Canadians think. Toronto: Canada Health Infoway; 2021:1–45. Available: [https://www.infoway-inforoute.ca/en/component/edocman/4011-canadian-digital-health-survey-2021-what-canadians-think/view-document?Itemid=0](https://www.infoway-inforoute.ca/en/component/edocman/4011-canadian-digital-health-survey-2021-what-canadians-think/view-document?Itemid=0) (accessed 2023 Apr. 10).
    
    

3.  Glazier RH, Green ME, Wu FC, et al. Shifts in office and virtual primary care during the early COVID-19 pandemic in Ontario, Canada. CMAJ 2021;193:E200–10.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMDoiMTkzLzYvRTIwMCI7czo0OiJhdG9tIjtzOjIzOiIvY21hai8xOTUvNDMvRTE0NjMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

4.  Canadians’ health care experiences during COVID-19: uptake of virtual care. Toronto: Canada Health Infoway; 2022. Available: [https://www.infoway-inforoute.ca/en/component/edocman/3828-canadians-health-care-experiences-during-covid-19/view-document?Itemid=0](https://www.infoway-inforoute.ca/en/component/edocman/3828-canadians-health-care-experiences-during-covid-19/view-document?Itemid=0) (accessed 2023 Apr. 10).
    
    

5.  COVID-19’s impact on emergency departments. Ottawa: Canadian Institute for Health Information; 2021. Available: [https://www.cihi.ca/en/covid-19-resources/impact-of-covid-19-on-canadas-health-care-systems/emergency-departments](https://www.cihi.ca/en/covid-19-resources/impact-of-covid-19-on-canadas-health-care-systems/emergency-departments) (accessed 2023 Apr. 10).
    
    

6.  2021 Canadian digital health survey: virtual visits. Toronto: Canada Health Infoway; 2021. Available: [https://insights.infoway-inforoute.ca/virtual\_visits](https://insights.infoway-inforoute.ca/virtual_visits) (accessed 2023 Apr. 10).
    
    

7.  Virtual care in Canada: progress and potential — report of the Virtual Care Task Force. Ottawa: Canadian Medical Association; 2022. Available: [https://www.cma.ca/sites/default/files/2022-02/Virtual-Care-in-Canada-Progress-and-Potential-EN.pdf](https://www.cma.ca/sites/default/files/2022-02/Virtual-Care-in-Canada-Progress-and-Potential-EN.pdf) (accessed 2023 Apr. 10).
    
    

8.  Virtual urgent care services: FY2021/22 funding criteria. Toronto: Ontario Health; 2021. Available: [https://www.ontariohealth.ca/sites/ontariohealth/files/2021-08/FundingCriteria-VirtualUrgentCare.pdf](https://www.ontariohealth.ca/sites/ontariohealth/files/2021-08/FundingCriteria-VirtualUrgentCare.pdf) (accessed 2023 Apr. 10).
    
    

9.  Hall JN, Ackery AD, Dainty KN, et al. Designs, facilitators, barriers, and lessons learned during the implementation of emergency department led virtual urgent care programs in Ontario, Canada. Front Digit Health 2022;4:946734. doi:10.3389/fdgth.2022.946734.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.3389/fdgth.2022.946734&link_type=DOI) 

10. McLeod SL, Mondoux S, Hall JN, et al. Demographic characteristics, outcomes and experience of patients using virtual urgent care services from 14 emergency department led sites in Ontario. CJEM 2023;25:65–73.
    
    

11. Austin PC. Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity-score matched samples. Stat Med 2009;28:3083–107.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1002/sim.3697&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=19757444&link_type=MED&atom=%2Fcmaj%2F195%2F43%2FE1463.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000271409300008&link_type=ISI) 

12. Chu C, Cram P, Pang A, et al. Rural telemedicine use before and during the COVID-19 pandemic: repeated cross-sectional study. J Med Internet Res 2021;23:e26960. doi:10.2196/26960.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.2196/26960&link_type=DOI) 

13. Mehrotra A, Jena AB, Busch AB, et al. Utilization of telemedicine among rural medicare beneficiaries. JAMA 2016;315:2015–6.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1001/jama.2016.2186&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=27163991&link_type=MED&atom=%2Fcmaj%2F195%2F43%2FE1463.atom) 

14. Dullet NW, Geraghty EM, Kaufman T, et al. Impact of a university-based outpatient telemedicine program on time savings, travel costs, and environmental pollutants. Value Health 2017;20:542–6.
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fcmaj%2F195%2F43%2FE1463.atom) 

15. Reed ME, Huang J, Graetz I, et al. Patient characteristics associated with choosing a telemedicine visit vs office visit with the same primary care clinicians. JAMA Netw Open 2020;3:e205873. doi:10.1001/jamanetworkopen.2020.5873.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1001/jamanetworkopen.2020.5873&link_type=DOI) 

16. Toronto’s Virtual Emergency Department [home page]. Available: [https://www.torontovirtualed.ca/](https://www.torontovirtualed.ca/) (accessed 2023 Apr. 10).
    
    

17. Virtual urgent care. Toronto: The Hospital for Sick Children. Available: [https://www.sickkids.ca/en/emergency/virtual-urgent-care/](https://www.sickkids.ca/en/emergency/virtual-urgent-care/) (accessed 2023 Apr. 10).
    
    

18. Rose S, Hurwitz HM, Mercer MB, et al. Patient experience in virtual visits hinges on technology and the patient-clinician relationship: a large survey study with open-ended questions. J Med Internet Res 2021;23:e18488. doi:10.2196/18488.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.2196/18488&link_type=DOI) 

19. Stamenova V, Agarwal P, Kelley L, et al. Uptake and patient and provider communication modality preferences of virtual visits in primary care: a retrospective cohort study in Canada. BMJ Open 2020;10:e037064. doi:10.1136/bmjopen-2020-037064.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMjoiMTAvNy9lMDM3MDY0IjtzOjQ6ImF0b20iO3M6MjM6Ii9jbWFqLzE5NS80My9FMTQ2My5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

20. Nabieva K, McCutcheon T, Liddy C. Connecting unattached patients to comprehensive primary care: a rapid review. Prim Health Care Res Dev 2023;24:e19. doi:10.1017/S1463423623000099.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1017/S1463423623000099&link_type=DOI) 

21. Wallace K. ‘Staggering’ number of Ontario emergency department closures revealed by Star analysis. Toronto Star 2023 Feb. 21, updated 2023 July 19. Available: [https://www.thestar.com/news/canada/staggering-number-of-ontario-emergency-department-closures-revealed-by-star-analysis/article\_6d958c6f-033f-5195-9a6a-4398457f1097.html](https://www.thestar.com/news/canada/staggering-number-of-ontario-emergency-department-closures-revealed-by-star-analysis/article_6d958c6f-033f-5195-9a6a-4398457f1097.html) (accessed 2023 Apr. 10).
    
    

22. Shigekawa E, Fix M, Corbett G, et al. The current state of telehealth evidence: a rapid review. Health Aff (Millwood) 2018;37:1975–82.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1377/hlthaff.2018.05132&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fcmaj%2F195%2F43%2FE1463.atom) 

23. Andrews E, Berghofer K, Long J, et al. Satisfaction with the use of telehealth during COVID-19: an integrative review. Int J Nurs Stud Adv 2020;2:100008. doi:10.1016/j.ijnsa.2020.100008.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/j.ijnsa.2020.100008&link_type=DOI) 

24. Shah VV, Villaflores CW, Chuong LH, et al. Association between in-person vs telehealth follow-up and rates of repeated hospital visits among patients seen in the emergency department. JAMA Netw Open 2022;5:e2237783. doi:10.1001/jamanetworkopen.2022.37783.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1001/jamanetworkopen.2022.37783&link_type=DOI) 

25. Alexander GC, Tajanlangit M, Heyward J, et al. Use and content of primary care office-based vs telemedicine care visits during the COVID-19 pandemic in the US. JAMA Netw Open 2020;3:e2021476. doi:10.1001/jamanetworkopen.2020.21476.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1001/jamanetworkopen.2020.21476&link_type=DOI) 

26. Lapointe-Shaw L, Salahub C, Bird C, et al. Characteristics and health care use of patients attending virtual walk-in clinics in Ontario, Canada: cross-sectional analysis. J Med Internet Res 2023;25:e40267. doi:10.2196/40267.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.2196/40267&link_type=DOI) 

27. Li KY, Ng S, Zhu Z, et al. Association between primary care practice telehealth use and acute care visits for ambulatory care-sensitive conditions during COVID-19. JAMA Netw Open 2022;5:e225484. doi:10.1001/jamanetworkopen.2022.5484.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1001/jamanetworkopen.2022.5484&link_type=DOI) 

28. Shaw J, Brewer LC, Veinot T. Recommendations for health equity and virtual care arising from the COVID-19 pandemic: narrative review. JMIR Form Res 2021;5:e23233. doi:10.2196/23233.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.2196/23233&link_type=DOI) 

29. Moe J, Wang EY, McGregor MJ, et al. Subgroups of people who make frequent emergency department visits in Ontario and Alberta: a retrospective cohort study. CMAJ Open 2022;10:E232–46.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiY21ham8iO3M6NToicmVzaWQiO3M6OToiMTAvMS9FMjMyIjtzOjQ6ImF0b20iO3M6MjM6Ii9jbWFqLzE5NS80My9FMTQ2My5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

30. Emergency Department Return Visit Quality Program. Toronto: Health Quality Ontario. Available: [https://www.hqontario.ca/Quality-Improvement/Quality-Improvement-in-Action/Emergency-Department-Return-Visit-Quality-Program](https://www.hqontario.ca/Quality-Improvement/Quality-Improvement-in-Action/Emergency-Department-Return-Visit-Quality-Program) (accessed 2023 Apr. 10).
    
    

31. Chartier LB, Jalali H, Seaton MB, et al. Qualitative evaluation of a mandatory provincial programme auditing emergency department return visits. BMJ Open 2021;11:e044218. doi:10.1136/bmjopen-2020-044218.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMjoiMTEvNC9lMDQ0MjE4IjtzOjQ6ImF0b20iO3M6MjM6Ii9jbWFqLzE5NS80My9FMTQ2My5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=)