Effectiveness Of Geriatric Rehabilitation In Inpatient And Day Hospital Settings: A Systematic Review And Meta-analysis

Introduction

Approximately 2.7 million newborns die every year, many from preventable or treatable conditions including prematurity, infection and birth asphyxia.1 Many low- and middle-income countries (LMIC) are struggling to meet the Every Newborn Action Plan target of fewer than 12 newborn deaths per 1000 live births by 2030.2 Health ministries urgently need approaches to upskill frontline healthcare workers to provide care to sick newborns.

The poor newborn outcomes in LMICs are due to many factors. Healthcare workers have limited skills and poor adherence to clinical guidelines; they also face large patient loads and high staff turnover, which make training difficult to implement.3–5 Clinical decision support system (CDSS) offers features that address many of these gaps in care delivery including improving fidelity to clinical guidelines and reducing medical errors.6–8 CDSS also has the potential to surmount training challenges. Incorporated into daily practice, CDSS offers the prospect of improvement that endures over time and across new and frequently changing staff. Several CDSS platforms have demonstrated efficacy in improving knowledge and confidence of frontline healthcare workers in neonatal care delivery.9–12

While CDSS is generally designed to be easy for people with smartphones and other computer experience to learn, fluency with modern technology cannot be assumed in low-resource settings. Indeed, the key challenge in scaling CDSS is that implementation requires initial training that has historically been done by information technology specialists or product designers, who are short in supply in low-income countries. Task shifting13 of training roles and other clinical work to lower cadre healthcare workers has been recognised as a sustainable and cost-effective method to expand access to medical care.14–20 This approach could provide a sustainable and successful implementation of CDSS. However, there have been no studies examining whether tasks traditionally done by specialists related to CDSS in low-resource settings can be task shifted to healthcare workers.

The NoviGuide Neonatal Essentials application (hereafter, NoviGuide) is a CDSS with six pathways for common neonatal encounters.21 These pathways include: (1) initial assessment <24 hours, (2) initial assessment >24 hours, (3) rounding, (4) spot check, (5) discharge and (6) seizure and abdominal emergency. NoviGuide contains instructional videos from Global Health Media Project22 depicting physical signs of common newborn conditions, breastfeeding guidance and newborn procedures. NoviGuide works both online and offline and can be deployed as a web or mobile application. It is configurable at the level of the facility to variations in equipment or protocols and its dashboards can show site-specific use. NoviGuide has been shown to improve the nurse-midwives’ knowledge and confidence to care for newborns at a rural general hospital in eastern Uganda.6 23

Given the limited evidence for a sustainable and scalable adoption of CDSS,24–26 our aim was to evaluate whether CDSS expert roles could be task shifted to midwives in the implementation of NoviGuide. First, we developed the NoviGuide Neonatal Essentials Trainer programme, a midwife-led training programme for the introduction of NoviGuide in health facilities. We then applied the Kirkpatrick model27 to evaluate: (1) the acceptability of midwives as trainers of NoviGuide among nurse-midwives caring for newborns at four rural health facilities in eastern Uganda, and (2) the effectiveness of the training programme on nurse-midwives’ uptake of NoviGuide and their perception of quality of newborn care in the four rural health facilities.

Methods

Study design

We conducted a 20-month, concurrent triangulation mixed-methods open cohort study from September 2020 to May 2022. We chose an open cohort study design to enrol all nurse-midwives following staff changes, capturing real-world situations with high attrition rates for NoviGuide implementation. We used Kirkpatrick’s programme evaluation model.27 We evaluated acceptability at level 1 (Reaction) for participant’s reaction immediately after the initial training, and at level 3 (Behaviour) for the participant’s attitude towards midwives as NoviGuide Trainers at 3 and 6 months. We evaluated effectiveness at level 2 (Learning) for participants’ newly acquired knowledge and skills to use NoviGuide, and at level 3 (Behaviour) for participant’s uptake of NoviGuide and perception of newborn care following the introduction of NoviGuide (table 1).

Table 1

Four evaluation levels of Kirkpatrick model and the data sources

Study site selection and setting

We selected four government-owned health facilities located in Tororo district, eastern Uganda, where newborn care is primarily provided by nurses and midwives. The sites include Tororo General Hospital and Mulanda, Nagongera and Mukujju Health Center (HC) IVs. These health facilities serve a population of 583 400 people.28 Tororo General Hospital conducts approximately 400 deliveries monthly and admits nearly 100 sick newborns per month from the community, health centres, private facilities and across the Kenya-Uganda border. At the launch of the study, Tororo General Hospital had 22 midwives providing both maternal and newborn care in the labour suite, postnatal ward and a small kangaroo mother care room. Staff work 2–3 per shift on each of these units with supervisory support from two to three medical officers. They rotate to other units at least annually. The kangaroo mother care room was refurbished in September 2021 to a neonatal unit with new equipment including neonatal incubators, phototherapy, warmers and oxygen concentrators. Six staff were assigned neonatal roles to work on this unit. The three HC IVs conduct between 70 and 130 deliveries each per month and had no dedicated area for sick newborn admissions. Six to eight midwives provide newborn care in addition to maternity care including labour and delivery, postnatal care and outpatient department. Staff work 1–2 per shift with supervisory support from one medical officer.

NoviGuide Neonatal Essentials Trainer programme

We trained two midwives as NoviGuide Trainers, tasked with instructing staff on the basic operation and troubleshooting of NoviGuide. The trainers attended three 3-hour sessions conducted at Tororo General Hospital boardroom.

There were three key elements to the training. First, trainers learnt the installation and set-up of the software onto tablets. This includes downloading the application, linking the software to a specific clinic using a 9-digit code and creating and managing user profiles.

Second, the trainers learnt how to train fellow nurse-midwives on the (1) basic function of NoviGuide, including its content and location of key functionalities such as contextual drug dosing calculators and preterm feeding widgets, (2) introduction to the use of the tablet, (3) value of NoviGuide in promoting fidelity to neonatal guidelines to reduce medical errors, (4) key safety considerations including what to do when one’s clinical judgement does not align with software information and (5) gamification features to track their use of the application.

Third, the trainers learnt how to monitor sites using the dashboards. This training includes review of dashboards, evaluation of NoviGuide adoption and address low uptake, data synchronisation and troubleshooting common technical issues.

We taught these three elements using a combination of PowerPoint presentations, role-play and support supervision. The trainers used the same PowerPoint presentations during the on-site training of nurse-midwives. In this way, the training was first modelled for the trainers.

Enrolment and training of the study participants

All nurse-midwives at the four study sites were invited to attend an introductory session at their respective sites. Eligibility criteria included provision of newborn care at the study sites, more than 18 years of age, having an active practising licence to practise and willingness to participate in the study. We obtained written informed consent before participation. We conducted recruitment first at Tororo General Hospital in September 2020 and at Mulanda, Nagongera and Mukujju HC IVs in March, April and May 2021, respectively. The delay was due to COVID-19 pandemic travel restrictions.

Following recruitment, NoviGuide Trainers travelled to each site and trained participants for 3 hours. The trainers used the PowerPoint presentations of the NoviGuide Neonatal Essentials Trainer programme. The trainers then provided technical support for NoviGuide use and addressed any technical problems daily during the first 2 weeks and monthly thereafter. As an open cohort study, the trainers also trained newly recruited participants following staff changes.

Introduction of NoviGuide at the health facilities

The NoviGuide Trainers introduced NoviGuide at the four study sites, adding each participant into the application as a user with a distinct password. Each site received two to four tablets (Amazon Fire HD 8 tablet) loaded with NoviGuide. The number of tablets depended on the number of staff per shift. With two to three staff per shift, the general hospital received four tablets, while the HC IVs each received two tablets because they have one to two staff per shift. Tablets were stored in lockable wooden cabinets located in the midwives’ office, neonatal unit or labour suite.

The trainers identified a key contact person (NoviNurse) at each site who kept the study team informed about any challenges encountered. The NoviNurse assisted with data synchronisation by intermittently connecting the tablets to WiFi provided by a MiFi modem (Airtel 4G modem model MF927U).

Data collection

At baseline, participants completed a questionnaire that included demographic information (age and sex), years of clinical experience, role (nurse or midwife), health facility, devices personally owned and access to internet.

Immediately after the training, participants completed a survey adapted from the Training Acceptability Rating Scale evaluating acceptability (Kirkpatrick level 1—Reaction). The first section29 consists of six statements assessing general acceptability, appropriateness and perceived effectiveness, negative side effects, consistency and social validity of midwives as NoviGuide Trainers. The participants rated each of the statements on a 6-point Likert scale indicating their degree of agreement or disagreement with responses 1–6, where 1 represents ‘Strongly disagree’ and 6 represents ‘Strongly agree’. The second section30 consists of nine statements assessing the participants’ perception about the training process and competence of Trainers. The participants rated each of the statements on a 4-point scale from 1 to 4 where 1 represents ‘Not at all’ and 4 ‘A great deal’.

At 3 and 6 months, participants completed another survey adapted from the Electronic Health Record End User survey and participated in focus group discussions for evaluation of both effectiveness and acceptability (Kirkpatrick level 2—Learning and level 3—Behaviour). We chose 3 and 6 months as these timepoints were close enough to the initial training for participants to remember their perception about the programme. The survey consists of 11 statements, including: ‘I received adequate training from the Trainers on how to use NoviGuide’, ‘My questions about NoviGuide were sufficiently answered by the Trainers’ and ‘The Trainers were able to provide technical support on NoviGuide when I needed it’. The respondents indicated their degree of agreement or disagreement on a 5-point Likert scale, where 1 represents ‘strongly disagree’ and 5 ‘strongly agree’. In total, we conducted 10 focus group discussions; four at the general hospital because of the large number of participants and two at each of the three HC IVs. Seven to eleven participants attended each discussion. Two female members of the study team guided the discussions. One moderated the discussions using the following questions: (a) Share your experience of caring for newborns before and after the introduction of NoviGuide, and (b) Describe your attitude towards midwives as NoviGuide Trainers. The other managed the audio recording. Each focus group opened with a statement explaining the purpose of the discussion and an assurance of confidentiality. All the interviews were conducted in English, audio recorded and lasted approximately 1 hour. The focus group data were transcribed verbatim, labelled with a unique number and kept on a password-protected computer.

At least once every week for 20 months, NoviNurses connected the tablets to WiFi by switching on modems to transfer NoviGuide usage data to a secure cloud-based database. We viewed the usage data from the site dashboards for the participant’s uptake of NoviGuide to further evaluate effectiveness (Kirkpatrick level 3—Behaviour).

Data analysis

We used descriptive statistics to analyse the participant’s baseline characteristics and NoviGuide uptake. For the Training Acceptability Rating Scale 1 and 2, we determined the mean scores and SD of each of the statements. For each of the Electronic Health Record End User survey statements, we calculated the median scores and IQRs. We then used a two-tailed Wilcoxon signed-rank test to compare the median scores at 3 and 6 months. To determine the magnitude of the differences, we calculated effect size using Cohen’s d. A positive effect size indicated an increase in the mean score while a negative effect size indicated a decrease. We considered effect sizes of 0.2 to <0.5 as small, 0.5 to <0.8 as medium and 0.8 and above as large.31 To assess uptake, we determined the proportion of trained participants who used NoviGuide following the training. We also captured the total number of newborn assessments entered into NoviGuide by each participant, expressing the results in a figure and determined the total and range of assessments per site. We analysed all quantitative data using Stata V.16 (StataCorp, College Station, Texas) setting the CI at 95% and considered p value <0.05 significant.

Qualitatively, we employed thematic analysis32 using Qualitative Data Analysis Miner Lite V.2.0.9 (Provalis Research, Montreal, Quebec, Canada). MKM cleaned the data by reading each transcript while listening to the original recording. Then, MKM and JA analysed the data. During the coding meetings, they developed subthemes emerging from the codes, further categorising these subthemes into three overarching themes: one theme in level 2 (Learning)—Newly gained knowledge and skills, and two themes in level 3 (Behaviour)—Changes in newborn care practices and Attitude towards midwives as trainers. In total, 10 subthemes emerged. For each subtheme, we included key illustrative quotes and examined for similarities and differences across study sites. The whole study team approved the finalised categorisation of the subthemes.

We concurrently triangulated the quantitative and qualitative data by assessing focus group discussion data for content areas that explained or contradicted survey data and NoviGuide usage data. We used Strengthening the Reporting of Observational Studies in Epidemiology checklist33 for cohort studies during the preparation of this manuscript.

Patient and public involvement

Neither patients nor the general public were involved in the design or management of the study.

Results

Participant characteristics

We screened 49 female nurse-midwives and enrolled them all with a mean age of 34 (range: 24–56) years. None declined to participate in the study. Of 49 participants, 26 (53.1%) worked at Tororo General Hospital and 8 (16.3%), 8 (16.3%) and 7 (14.3%) at Mulanda, Mukujju and Nagongera HC IVs, respectively. We first enrolled participants in September 2020 from Tororo General Hospital (13/26) before adding Mulanda, Nagongera and Mukujju HC IVs in March, April and May 2021, respectively. This was an open cohort with new participants enrolled during the course of the study following staff changes. Majority (46/49 (94%)) were midwives and only 3/49 (6%) were nurses, 23/49 (46.9%) owned smartphones but only 12/49 (24.5%) reported accessing the internet daily. Of 49 participants, 21 (42.7%) had work experience of 3–10 years, 12 (24.5%) had worked for 11–20 years, 11 (22.5%) for 0–2 years and only 5 (10.2%) for more than 21 years.

Of 49 enrolled participants, only 17 (35%) remained in the study at the time of closure because of staff changes, including: (a) transfer to other wards within or to other health facilities by the district administration, and (b) refurbishment of the neonatal unit at Tororo General Hospital in September 2021, where only six participants were assigned to care for newborns. The rest were given non-neonatal assignments.

Kirkpatrick level 1: Reaction

Immediately following the initial training, all participants reported high acceptability of midwives as trainers of NoviGuide (mean 5.9, SD 0.37) (table 2). They perceived the use of midwives as NoviGuide Trainers an appropriate approach that would result in increased interest in NoviGuide among staff. The participants perceived the training by midwives sufficient for them to develop the skills needed to use NoviGuide during their care of newborns (mean 3.88, SD 0.33). The participants felt confident to use NoviGuide (mean 3.80, SD 0.41). The trainers were perceived as competent, motivating and able to relate well with the participants during the training sessions.

Table 2

The Training Acceptability Rating Scale (Kirkpatrick evaluation level 1—Reaction)

·      *1: Strongly disagree, 2: Moderately disagree, 3: Slightly disagree, 4: Slightly agree, 5: Moderately agree, 6: Strongly agree.

·      †1: Not at all, 2: A little, 3: Quite a lot, 4: A great deal.

Kirkpatrick level 2: Learning

Three subthemes emerged concerning the participants’ newly gained knowledge and skills, namely: (1) learnt how to use NoviGuide, (2) gained technology skills and (3) colleagues learnt and helped others (table 3). Across all study sites, participants reported that the training was sufficient for them to start using NoviGuide. They discussed that NoviGuide was easy to learn because of the understandable language and guidance provided by the software. Some of the participants, especially those who did not own smartphones, reported gaining new skills in using technology. In addition to the Trainer’s support, participants also reported receiving support from their colleagues, especially during weekend and night shifts. Support sought included: (a) how to navigate the tablet, (b) waking an unresponsive tablet and (c) training colleagues who were away during the initial training.

·      Collapse inline 

·      View popup

Table 3

Focus group discussion themes and key illustrative quotes (Kirkpatrick evaluation levels 2 and 3)

·      MUK, Mukujju Health Center IV; MUL, Mulanda Health Center IV; NAG, Nagongera Health Center IV; TOR, Tororo General Hospital.

The participants used NoviGuide for varying newborn conditions. Of all the assessments made into NoviGuide, 13.8% (558/4045) were for preterm newborns, 17.5% (709/4045) for newborns weighing under 2.5 kg and 21.1% (855/4045) had a temperature less than 36.5°C.

In addition to NoviGuide use during the admission of newborns to the hospital, participants used NoviGuide for rounding (12.2% (492/4045)) and during discharge (9.3% (376/4045)), where they used a checklist to assess readiness for discharge. The participants also viewed instructional videos within NoviGuide. These videos, as discussed during the focus groups, were used especially for first time mothers and as a quick reminder of clinical skills. Of the 44 participants whose use of NoviGuide videos was assessed, 25 (56.8%) watched breastfeeding videos, 10 (22.7%) a discharge video and 9 (20.5%) a video on danger signs. For reminders on clinical skills, 22 (50%) watched a resuscitation video, 16 (36.4%) an intravenous insertion video and 12 (27.3%) a video on how to insert a nasogastric tube.

Discussion

In our study, the participants revealed a high degree of acceptance of midwives as NoviGuide Trainers. The midwife-led training programme resulted in high uptake of NoviGuide among participants and perceived improvement in newborn care practices.

Task shifting has been shown as a cost-effective strategy for addressing staff shortages in the provision of high-quality care for chronic medical conditions.34–36 Most studies on neonatal CDSS in Africa, including our previous research, evaluated the acceptability and feasibility of individual products.6 37 38 This study contributes to the limited evidence essential for the sustainable scale-up of CDSS in similar settings. However, CDSS can also be considered a health system strengthening intervention requiring changes at the individual, facility and health system levels.26 39 40 Successful adoption of complex interventions requires workflow adjustments at different levels of the health systems while tackling drug shortages, frequent staff transfers, heavy workload and staff training and support. Additional resources are needed for CDSS implementation including internet coverage, reliable electricity and mobile devices which may not be easily available in rural settings.

Characteristics of the CDSS should also be considered for adoption success.9 Our findings of improved confidence and newborn care practices are consistent with other studies on CDSS.7–10 However, contrary to other studies39 41 where poor computer skills are reported as a barrier to implementation of CDSS in African settings, lack of prior exposure to smartphone use was not a hindrance to NoviGuide use in our study. Uptake of the software was very high. We attribute this to the usability and unique functionalities of NoviGuide. NoviGuide uses simple terminology and provides easy-to-follow guidance at the point of care. To further improve care, usage data could potentially be used to support quality improvement initiatives and staff supervision. Further research is therefore required to evaluate how site dashboards could be integrated into routine organisational systems.

Our results also aligned with studies42–44 where midwives effectively took on additional roles as trainers for facility-based interventions. However, it is important to put into consideration the implications that task shifting has on other roles of the midwife trainers. This is because trainers are removed from clinical care to prepare for and provide training. A cost-effectiveness evaluation is needed to compare this approach to conventional methods.

Our study has several limitations. To avoid selection bias, we enrolled all nurses and midwives caring for newborns at the respective health facilities. However, two trainers in our programme had previous experience with NoviGuide; this likely contributed to the effectiveness of our training programme. New trainers would likely take additional time for training. However, the idea of using users as trainers is an appealing strategy for scaling CDSS. Our work describes a task-shifting approach from CDSS experts to midwives. We did not formally evaluate the implications that task shifting had displacing other administrative roles of the midwives. While the universally positive opinions from participants about the programme are encouraging, it raises concerns about reporting bias; it is possible that participants felt obligated to give their supervisors positive reviews as trainers. However, the CDSS data suggest that participants sought to use the CDSS even outside of the presence of the trainers. One site had recently undergone refurbishment of their neonatal unit. The perception of better care resulting from new resources at that facility could have positively biased participants’ perception of improved quality of newborn care. We evaluated outcomes at Kirkpatrick levels 1–3. Therefore, further research at level 4 is required for long-term impact of the training programme on newborn outcomes and organisational systems.

Conclusion

The use of midwives as NoviGuide Trainers was acceptable in the introduction of a complex neonatal CDSS among nurse-midwives at four rural health facilities in eastern Uganda. The trained midwives provided technical support and NoviGuide troubleshooting. This support resulted in high uptake of NoviGuide among nurse-midwives and improved confidence and self-reported improvements in neonatal care timeliness, accuracy and team communication. Task shifting information technology roles to midwives could play a key role in the scale-up of CDSS. However, resources including internet coverage, reliable electricity and mobile devices should be considered for sustainable scale-up in low-resource settings. Further research is also required on the cost-effectiveness and long-term impact on newborn outcomes and organisational systems.

Data availability statement

Data are available upon reasonable request.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

This study involves human participants and was approved by Makerere University School of Biomedical Sciences (SBS-717), Uganda National Council for Science and Technology (HS 2739) and Western Institutional Review Board (20200047). All participants provided written informed consent. Participants gave informed consent to participate in the study before taking part.

Acknowledgments

We acknowledge all the parents and the caregivers who allowed their newborns to be reviewed by the nurse-midwives, using NoviGuide, during the course of the study. We thank the two midwives, Sr Atim Grace and Sr Beatrice Musungu, whom we trained as NoviGuide Trainers and all the nurse-midwives from the four sites who took part in the study. We are also grateful to Dr Okoth Obbo, the district health officer and the administrative teams at the four sites for their generous support before and during the study period. We also acknowledge Sr Daisy Sikola Wandera, a retired midwife, who reviewed the training materials and assisted the principal investigator during the execution of the study.

References

1.     Beard JR, Officer A, de Carvalho IA, Sadana R, Pot AM, Michel JP, et al. The world report on ageing and health: a policy framework for healthy ageing. Lancet. 2016;387:2145–54.

Article PubMed Google Scholar 

2.     Partridge L, Deelen J, Slagboom PE. Facing up to the global challenges of ageing. Nature. 2018;561:45–56.

Article CAS PubMed Google Scholar 

3.     Lu AT, Quach A, Wilson JG, Reiner AP, Aviv A, Raj K, et al. DNA methylation GrimAge strongly predicts lifespan and healthspan. Aging (Albany NY). 2019;11:303–27.

Article CAS PubMed Google Scholar 

4.     Levine ME, Lu AT, Quach A, Chen BH, Assimes TL, Bandinelli S, et al. An epigenetic biomarker of aging for lifespan and healthspan. Aging (Albany NY). 2018;10:573–91.

Article PubMed Google Scholar 

5.     Belsky DW, Caspi A, Arseneault L, Baccarelli A, Corcoran DL, Gao X, et al. Quantification of the pace of biological aging in humans through a blood test, the DunedinPoAm DNA methylation algorithm. Elife. 2020;9:e54870.

Article CAS PubMed PubMed Central Google Scholar

Background

There is good evidence from randomised controlled trials [1] and real-world studies [2, 3] that e-cigarettes help people to quit smoking. E-cigarette use is behaviourally similar to cigarette smoking and the devices deliver nicotine effectively [4]. They are generally regarded as much less harmful than combustible tobacco, but pose some risks compared with neither smoking nor vaping [5]. The extent to which vaping protects against or increases the risk of relapse to smoking in the longer term is not yet clear. Given ex-smokers represent a growing proportion of the adult population [6], monitoring patterns of vaping among people who have quit smoking is important because unless vaping prevents relapse to smoking, it will expose users to some level of additional harm [5].

E-cigarettes were first introduced to the UK market in 2008. They were rarely used up to 2011 [7], then rapidly became popular as a method of quitting smoking. Up to mid-2011, fewer than one in 100 quit attempts in England involved the use of an e-cigarette; by 2014 this number had risen to more than one in four [8]. In the years that followed, the proportion of ex-smokers who had been quit for ≥ 1 year who reported current vaping increased steadily, from 3.3% in 2014 to 10.4% in 2019 [9]. While most ex-smokers who vape likely start using e-cigarettes while smoking and continue to vape beyond a successful quit attempt, some appear to take up vaping after stopping smoking. Between 2014 and 2019, 7.1% of ex-smokers who had been quit for < 1 year who did not use an e-cigarette in their quit attempt reported current vaping, as did an increasing proportion of ex-smokers who quit before e-cigarettes became popular in 2011 (from 0.8% in 2014 to 2.1% in 2019) [9].

Since 2021, there has been a substantial increase in vaping in England among all smoking statuses, which appears to have been linked to the introduction of new disposable e-cigarettes [10, 11]. The proportion of ≥ 1y ex-smokers who reported having vaped for more than 6 months doubled between the start of 2021 and October 2023 (from 8 to 16%) [10] and the proportion currently using disposable e-cigarettes increased from 0 to 4% [11]. Studies show the recent increase in vaping has been much greater among younger adults and those who drink more heavily [10,11,12]. It is not clear whether the same patterns have occurred among ex-smokers specifically, or whether there have been differences by other key sociodemographic characteristics (e.g., gender or socioeconomic position). It is also not clear to what extent the increase in vaping prevalence among ex-smokers reflects more people taking up vaping after smoking cessation or a change in the types of ex-smokers who are vaping.

This study aimed to examine the extent to which there has been an increase over time in vaping among adults in England who have quit smoking and when uptake of vaping takes place in relation to cessation. Specifically, we analysed trends in vaping prevalence among smokers trying to quit and among ex-smokers who (i) quit ≥ 1 year ago, (ii) quit recently and did not use an e-cigarette to do so, and (iii) quit before e-cigarettes became popular. We also explored how changes in the prevalence of vaping among ex-smokers differed according to their sociodemographic characteristics and level of alcohol consumption, and whether their profiles (in terms of their duration of abstinence and sociodemographic, drinking, and vaping characteristics) have differed since disposable e-cigarettes started to become popular.

Pre-registration

The study protocol and analysis plan were pre-registered on Open Science Framework (https://osf.io/87tkw/). We amended our planned analyses of trends in recent uptake of vaping after smoking cessation due to the small sample available for this outcome (see statistical analysis section for details).

Design

We analysed data from the Smoking Toolkit Study, a representative repeat cross-sectional survey of adults in England [13]. The survey began in November 2006 and is ongoing. Each month, a new sample of approximately 1,700 adults is selected via a hybrid of random probability and simple quota sampling. Data were collected face-to-face up to the start of the Covid-19 pandemic and have been collected via telephone interviews since April 2020; the two modalities show good comparability on key sociodemographic and nicotine use indices [14].

The present analyses focused on data from ex-smokers surveyed between October 2013 (the first wave to assess vaping status among ≥ 1y ex-smokers) and May 2024 (the most recent data at the time of analysis).

Detailed questions on vaping (beyond current use and use in quit attempts) were included in the survey from July 2016, so we restricted the sample to those surveyed between July 2016 and May 2024 for analyses addressing changes in the profile of ex-smokers who vape. Vaping characteristics were not assessed in certain waves during this period (May/June/August/September/November/December 2022; February/March/May/August/September/November/December 2023; and February/March/May 2024), so analyses of these variables were limited to those surveyed in eligible waves.

Measures

Full details of the measures are provided in the study protocol (https://osf.io/87tkw/).

Smoking status

Smoking status was assessed by asking participants which of the following best applied to them: (a) I smoke cigarettes (including hand-rolled) every day; (b) I smoke cigarettes (including hand-rolled), but not every day; (c) I do not smoke cigarettes at all, but I do smoke tobacco of some kind (e.g., pipe, cigar or shisha); (d) I have stopped smoking completely in the last year; (e) I stopped smoking completely more than a year ago; or (f) I have never been a smoker (i.e., smoked for a year or more). Those who responded a-d were considered past-year smokers. Those who responded d were considered < 1y ex-smokers and those who responded e were considered ≥ 1y ex-smokers.

Main outcomes

Use of e-cigarettes in quit attempts was assessed among past-year smokers who made at least one attempt to stop smoking in the past year. Quit attempts were assessed with the question: ‘How many serious attempts to stop smoking have you made in the last 12 months? By serious attempt I mean you decided that you would try to make sure you never smoked again. Please include any attempt that you are currently making and please include any successful attempt made within the last year’. Those who reported having made at least one quit attempt were then asked: ‘What did you use to help you stop smoking during the most recent serious quit attempt?’ Those who responded ‘electronic cigarette’ were considered to have used an e-cigarette to support their quit attempt.

Current vaping among ≥ 1y ex-smokers was assessed with the question: ‘Can I check, are you using any of the following?’. Those who reported using an e-cigarette were considered current vapers.

Recent uptake of vaping after smoking cessation was defined as current vaping among < 1y ex-smokers who did not use e-cigarettes in their most recent quit attempt [9], assessed with the question: ‘Can I check, are you using any of the following either to help you stop smoking, to help you cut down or for any other reason at all?’. Those who reported using an e-cigarette were considered current vapers.

Late uptake of vaping after smoking cessation was defined as current vaping among people who quit smoking before e-cigarettes became popular in 2011 [9]. Data were not collected on the timing of vaping uptake, or on the use of e-cigarettes in quit attempts that occurred more than a year ago, so we were unable to identify ex-smokers who had quit since 2011 without using e-cigarettes and therefore could not include them in analyses of this outcome. Duration of abstinence (i.e., how many years ago a participant quit smoking) was calculated as the participant’s actual age minus the age when they stopped smoking. We identified those who quit smoking before 2011 from the year in which they were surveyed and duration of abstinence (e.g., participants surveyed in 2013 with at least 3 years of abstinence, 2014 with at least 4 years of abstinence, 2015 with at least 5 years of abstinence, etc.). Current vaping was assessed as described above, with the question: ‘Can I check, are you using any of the following?’.

Participant characteristics

Sociodemographic characteristics included age, gender, and occupational social grade (ABC1 includes managerial, professional, and upper supervisory occupations, C2DE includes manual routine, semi-routine, lower supervisory, state pension, and long-term unemployed).

Past-6-month alcohol consumption was assessed with the three-item AUDIT-C. Scores range from 0–12, with higher scores indicating higher levels of consumption. A score of 0 indicates that the participant is a non-drinker, ≤ 4 is considered low-risk, ≥ 5 increasing and higher risk, and ≥ 11 possible dependence. Data on alcohol consumption were only available from March 2014, so analyses by alcohol consumption were limited to this period.

Vaping characteristics included vaping frequency, duration, main device type, usual nicotine strength, and usual source of purchase.

Statistical analysis

Data were analysed using R v.4.2.1. The Smoking Toolkit Study uses raking to weight the sample to match the population in England [13]. The following analyses used weighted data. We excluded participants with missing data on smoking or vaping status. Missing cases on other variables were excluded on a per-analysis basis.

Overall trends in vaping prevalence and uptake among ex-smokers

We used logistic regression to estimate trends across the study period in (i) use of e-cigarettes in attempts to stop smoking, (ii) current vaping among ≥ 1y ex-smokers, (iii) recent uptake of vaping after smoking cessation among < 1y ex-smokers who did not use e-cigarettes in their quit attempt, and (iv) late uptake of vaping after smoking cessation among people who quit smoking before 2011.

Time was modelled using restricted cubic splines, to allow for flexible and non-linear trends. For outcomes (i), (ii), and (iv), we modelled trends by survey month (splines with five knots). We had intended to do the same for recent uptake of vaping after smoking cessation, but sample sizes in each monthly survey wave were too small (mean [SD] monthly number of < 1y ex-smokers who did not use e-cigarettes to quit = 14.0 [6.3]; mean [SD] number who vaped = 1.1 [1.2]). We therefore aggregated data annually (12-month periods from October to the following September; e.g., 2013/14 = October 2013 to September 2014, etc.) for this outcome and reduced the number of knots to three so as not to overfit the modelled trend to the reduced number of datapoints. We used predicted estimates from the models to plot trends across the study period.

In a planned sensitivity analysis, we repeated the model for late uptake of vaping after smoking cessation with a restricted sample. We included only those with ≥ 14 years of abstinence (the minimum duration of abstinence for people who quit before 2011 and who were surveyed in 2024), to reduce the impact of this cohort’s duration of abstinence increasing across the study period (i.e., from ≥ 3 years for those surveyed in 2013 to ≥ 14 years for those surveyed in 2024).

Trends in vaping prevalence and uptake among subgroups of ex-smokers

To explore moderation of trends in vaping among (i) ≥ 1y ex-smokers and (ii) people who quit smoking before 2011 by age, gender, occupational social grade, and level of alcohol consumption, we repeated each model including the interaction between the moderator of interest and time – thus allowing for time trends to differ across subgroups. Each of the interactions was tested in a separate model. We did not model subgroup trends in recent uptake of vaping after smoking cessation (as planned) because of the small sample size.

Age and alcohol consumption (AUDIT-C) were modelled using restricted cubic splines with three knots (placed at the 5, 50, and 95% percentiles), to allow for non-linear relationships. We displayed estimates for specific ages (18-, 25-, 35-, 45-, 55-, and 65-year-olds) and AUDIT-C scores (0, 3, 6, 9, and 12) to illustrate how trends differ across ages and levels of alcohol consumption. Note that the models used to derive these estimates included data from participants of all ages and AUDIT-C scores.

Changes in the profile of ex-smokers who vape

We used descriptive statistics to compare the profiles of ≥ 1y ex-smokers who vaped, before and after disposable e-cigarettes started to become popular in England. Given vaping characteristics were not assessed before July 2016, we restricted this analysis to participants from this wave onwards. In line with evidence showing the rise in use of disposables started around June 2021 [15], we considered July 2016 to May 2021 to be the pre-disposables period and June 2021 to May 2024 to be the disposables period [16].

We reported data on quitting history (i.e., duration of abstinence), sociodemographic characteristics, alcohol consumption, and vaping characteristics. We calculated absolute percentage point changes (with 95%CIs) in the proportion belonging to each subgroup (avg_comparisons function, marginaleffects package [17]). In planned sensitivity analyses, we restricted the pre-disposables period to April-2020 to May-2021 (when data were consistently collected via telephone). Sample sizes were too small to repeat these analyses for ex-smokers who took up vaping after smoking cessation (recent uptake n = 115 [41/74 pre-disposables/disposables period]; late uptake n = 196 [59/137]). In an unplanned analysis, we explored changes in mean duration of abstinence among ≥ 1y ex-smokers who vaped in more detail, aggregating data annually across the entire study period (in 12-month periods from October to November the subsequent year, from October 2013 to May 2024) and modelling the trend using restricted cubic splines (three knots).

A total of 208,640 adults (≥ 18y) in England were surveyed between October 2013 and May 2024. We analysed data from 54,251 participants who reported having tried to stop smoking in the past year or having stopped smoking more than a year ago (weighted mean [SD] age = 49.2 [18.2] y; 46.9% women). A flow diagram showing the derivation of the subsamples used for each analysis is provided in Fig. 1 and characteristics of each subsample are summarised in Table 1.

Fig. 1

Trends in vaping prevalence and uptake

Across the study period, there were non-linear changes in the prevalence of use of e-cigarettes in attempts to stop smoking, current vaping among ≥ 1y ex-smokers, and recent and late uptake of vaping after smoking cessation (Fig. 2).

Fig. 2

Trends in the use of e-cigarettes for stopping smoking, current vaping among ex-smokers, and uptake of vaping after smoking cessation, October 2013 to May 2024. Panels show the prevalence of (A) e-cigarette use in quit attempts by past-year smokers (n = 12,593); (B) current vaping among ≥ 1y ex-smokers (n = 41,658); (C) current vaping among < 1y ex-smokers who did not use e-cigarettes in their most recent quit attempt (n = 1,782); and (D) current vaping among ≥ 1y ex-smokers who quit smoking before e-cigarettes started to become popular in 2011 (n = 29,029). Trends in (A), (B), and (D) were modelled monthly (restricted cubic splines; five knots); (C) was modelled annually (three knots) on account of small samples. Lines represent modelled weighted proportions. Shaded bands represent 95% confidence intervals. Points represent unmodelled weighted proportions

Use of e-cigarettes in attempts to stop smoking

Among past-year smokers who tried to quit (n = 12,593; 32.5% of the n = 38,814 past-year smokers surveyed), the proportion who reported using e-cigarettes to support their most recent quit attempt increased from 26.9% [24.0–30.0%] in October 2013 to 37.1% [35.1–39.1%] in July 2016. It then declined to 30.0% [28.1–31.9%] by August 2019 and remained stable for a short period (at an average of 29.6% [27.9–31.3%] between August 2019 and May 2021), before increasing from 30.2% [28.4–32.1%] to a new high of 41.4% [37.7–45.2%] between June 2021 and May 2024 (Fig. 2A).

Current vaping among ≥1y ex-smokers

Among ≥ 1y ex-smokers (n = 41,658), the proportion who reported current vaping increased from 1.9% [1.5–2.5%] in October 2013 to 9.6% [9.0–10.2%] in December 2017, was relatively stable between December 2017 and May 2021 (at an average of 10.1% [9.5–10.8%]), then increased further from 11.2% [10.6–11.9%] to 20.4% [18.7–22.2%] between June 2021 and May 2024 (Fig. 2B).

The increase in current vaping among ≥ 1y ex-smokers was greater at younger ages (e.g., reaching 58.9% among 18-year-olds vs. 10.7% among 65-year-olds; Fig. 3A; Additional File 1: Table S1) and among those with the highest levels of alcohol consumption (e.g., reaching 35.4% among those with an AUDIT-C score of 12; Fig. 3D; Additional File 1: Table S1). The proportion who vaped was consistently slightly higher among those from less compared with more advantaged social grades, but changes in prevalence over time were similar (Fig. 3C). There were no notable differences by gender (Fig. 3B).

Fig. 3

Trends in current vaping and late uptake of vaping after smoking cessation among subgroups of ex-smokers, October 2013 to May 2024. Panels show the prevalence of (i) current vaping among ≥ 1y ex-smokers and (ii) current vaping among ≥ 1y ex-smokers who quit smoking before e-cigarettes started to become popular in 2011 (i.e., late uptake of vaping after smoking cessation), by (A) age, (B) gender, (C) occupational social grade, and (D) level of alcohol consumption. Lines represent modelled weighted proportions (time modelled monthly using restricted cubic splines; five knots). Shaded bands represent 95% confidence intervals. *Estimates of late uptake of vaping after smoking cessation are not reported for 18- and 25-year-olds because very few participants in this age range could have quit smoking as an adult before 2011. **Alcohol consumption was assessed from March 2014 onwards. Estimates of prevalence in the first and last months of the time series are provided in Additional File 1: Table S1

Recent uptake of vaping after smoking cessation

Among < 1y ex-smokers who reported not using e-cigarettes in their most recent quit attempt (n = 1,782), there was an uncertain decline in the proportion who reported current vaping between 2013/14 and 2017/18, from 9.3% [6.3–13.5%] to 6.0% [4.5–7.9%], followed by an uncertain increase to 13.7% [9.7–19.0%] between 2017/18 and 2023/24 (Fig. 2C).

Late uptake of vaping after smoking cessation

Among ≥ 1y ex-smokers who quit smoking before e-cigarettes started to become popular in 2011 (n = 29,029), the proportion who reported current vaping increased from 0.4% [0.2–0.8%] in October 2013 to 1.9% [1.6–2.3%] in June 2021 then increased more rapidly, reaching 3.7% [2.8–4.9%] by May 2024 (Fig. 2D). The trend was similar when we restricted this group to ≥ 14y ex-smokers (to ensure a more consistent duration of abstinence within this group across the period), although absolute prevalence estimates were lower in earlier months: increasing from 0.01% [0.0–0.3%] in October 2013 to 1.1% [0.8–1.4%] in June 2021 and to 3.6% [2.6–5.0%] in May 2024 (Additional File 2: Fig. S1).

The increase in current vaping among ≥ 1y ex-smokers who quit smoking before e-cigarettes started to become popular in 2011 appeared more pronounced among those who were younger (e.g., reaching 10.1% among 35-year-olds vs. 3.4% among 65-year-olds; Fig. 3A) and those who drank more heavily (e.g., reaching 13.9% among those with an AUDIT-C score of 12; Fig. 3D), although 95% CIs were wide, introducing some uncertainty (Additional File 1: Table S1). There were also potentially smaller differences by gender and occupational social grade near the end of the period, with rates rising to higher levels among men than women (4.7% vs. 2.7%; Fig. 3B) and those from less vs. more advantaged social grades (5.2% vs. 2.7%; Fig. 3C), but these differences were uncertain (Additional File 1: Table S1).

Changes in the profile of ex-smokers who vape

There were several differences in the profile of ≥ 1y ex-smokers who vaped from before to after disposable e-cigarettes started to become popular. Results are reported in detail in Additional File 3. Briefly, changes included greater mean duration of abstinence, younger age, longer duration of vaping, greater use of disposable e-cigarettes and high-strength nicotine e-liquids, and a shift away from purchasing vaping products from vape shops towards supermarkets and convenience stores.

Over the past decade, there have been clear shifts in vaping prevalence and uptake among adults in England who have quit smoking. In October 2013, when e-cigarettes were still fairly new and delivered nicotine less efficiently, around one in 50 ≥ 1y ex-smokers vaped. This number increased steadily to one in ten by the end of 2017 and remained stable for several years. It then increased sharply from 2021, reaching one in five by May 2024, equivalent to approximately 2.2 million people (45.2 million adults ≥ 18y in England [18] * 23.8% ≥ 1y ex-smokers [Smoking Toolkit Study, January–May 2024] * 20.4% vaping prevalence). This pattern is consistent with that observed in the general adult population: an initial rise in popularity of e-cigarettes, followed by a plateau and then a subsequent rapid rise linked to the introduction of new disposable e-cigarettes to the market [10].

Much of this increase in vaping prevalence among ex-smokers is likely to be the result of more people using e-cigarettes as a smoking cessation aid who continue to use them after stopping smoking. The timing of the changes in vaping prevalence we observed coincided with changes in the prevalence of the use of e-cigarettes by people attempting to quit smoking. Studies have shown that a substantial proportion of those who quit with the support of an e-cigarette continue to vape for many months (and in some cases, years) beyond their successful quit attempt. For example, a randomised controlled trial of e-cigarettes vs. nicotine replacement therapy for smoking cessation found that among those in the e-cigarette condition who were abstinent at one year, 80% were still vaping [19]. UK guidance advises people not to rush to stop vaping after quitting smoking, but rather to gradually reduce their vaping frequency or nicotine strength when they feel confident that they can do this without going back to smoking [20]. As such, one would expect to see an increase in vaping among ex-smokers as use of e-cigarettes in quit attempts increases.

However, not all of the increase in vaping among ex-smokers was attributable to continued vaping after successfully quitting smoking with an e-cigarette. Our data also provide evidence of an increase in the uptake of vaping after successful smoking cessation. In October 2013, vaping was rare among people who quit smoking before e-cigarettes started to become popular in 2011, at around one in 250 ex-smokers. By May 2024, this number had increased to one in 27, equivalent to approximately 212,000 people (45.2 million adults ≥ 18y in England [18] * 12.7% ex-smokers who quit before 2011 [Smoking Toolkit Study, January–May 2024] * 3.7% vaping prevalence). When we set the minimum duration of abstinence constant across this period (at ≥ 14y) the increase was even more stark, with the number of vapers increasing from one in 10,000 to one in 27. Differences over time in the uptake of vaping among recent (< 1y) ex-smokers were uncertain, at least partially due to smaller sample sizes, but also suggested a possible increase in recent years.

Uptake of vaping among ex-smokers may be influenced by new product developments and social trends. Increases in vaping prevalence were greatest at younger ages, among whom disposable e-cigarettes (and as a result, vaping more generally) have become particularly popular since 2021 [11, 15]. These influences may be greater among those with a propensity for risk-taking behaviour [21]. Consistent with this, increases in vaping among ex-smokers were also larger among those who reported the highest levels of alcohol consumption. A similar pattern has been documented among adults who have never regularly smoked [12].

In terms of the profile of ex-smokers who vape, we observed several differences since disposable e-cigarettes started to become popular. Most of these reflect changes that have occurred among vapers more generally: younger age, increased use of disposables and higher nicotine strengths, and increased purchasing from supermarkets and convenience stores [11, 12, 22, 23]. Ex-smoking vapers surveyed more recently also reported a longer duration of abstinence from smoking, on average, than those surveyed earlier. This may partly reflect ex-smokers who vape accumulating over time as people take up vaping and continue to vape long-term (echoed by results indicating more are vaping for > 1 year). It may also reflect increased uptake of vaping among long-term ex-smokers or more recent ex-smokers relapsing back to smoking.

The health impacts of people taking up vaping after having stopped smoking will depend on what they would be doing if they did not vape. If they would otherwise not use nicotine, there is a risk that starting to vape may increase their risk of relapse to smoking by reintroducing them to regular nicotine exposure (although people typically report lower levels of dependence on vaping than smoking [24]). Vaping, while much less harmful than smoking, will also expose long-term ex-smokers to more harm than not vaping or smoking [5]. However, if ex-smokers take up vaping instead of relapsing to smoking this will reduce the harm they are exposed to [5]. Among very long-term ex-smokers, the risk of relapse would be low [25], so taking up vaping is probably more likely to have unintended consequences (i.e., exposure to harm, increased risk of relapse) than benefits. More research is needed to better understand the extent to which vaping increases vs. reduces the risk of relapse to smoking (both among ex-smokers who vape continuously from the point of a successful quit attempt and among those who take up vaping after quitting smoking) in different tobacco and nicotine regulatory contexts and markets. As with examining whether e-cigarettes act as a causal gateway to smoking among youth [26], this research should triangulate evidence from both the individual- and population-level using diverse methodologies with different sources of bias, and in priority groups that exhibit differential risks of returning to smoking.

The plateau in current vaping among long-term ex-smokers between 2018 and 2021 has a number of possible explanations. If people were quitting smoking with the use of e-cigarettes at a broadly constant rate, and continuing to vape long-term (with a proportion eventually quitting vaping too) at similar rates, then one would expect the proportion of long-term ex-smokers who were vaping to grow at a broadly linear rate, providing the rates of vaping uptake after cessation were also constant (which we observed during this period in the current study). Thus, the observed plateau between 2018 and 2021 (going against the previously observed steady increase) may reflect an increase during that period of long-term ex-smokers quitting vaping and/or relapsing to smoking. During that period, we also saw the average duration of smoking abstinence among long-term ex-smokers who vaped increase up to around 2019 but there appeared to be a plateau thereafter. In separate studies, we have observed a slowing in overall smoking prevalence around the same period [27], as well as increases in the proportion of non-daily smokers [28] and increases in the prevalence of the dual use of e-cigarettes and cigarettes [29]. All of which is consistent with increased relapse to non-daily smoking among long-term ex-smokers from around 2018 onwards. Insofar that this occurred, the cause(s) is unclear but coincided with big increases in the risk perceptions of e-cigarettes [30] and the onset of the covid pandemic and its associated impacts. When formulating vaping policy, any adverse effects on relapse to long-term ex-smokers who vape may represent a serious and unintended public health risk to be considered for countries in which large numbers of people have already switched from smoking to vaping. However, in the absence of direct evidence on changes in late relapse rates, it also remains possible that the increase in vaping among ex-smokers may offer some protection against relapse to smoking, and the changes described above are a consequence of other factors. More research into how relapse rates are changing in the context of changes in vaping prevalence, including following changes in the market and regulation of e-cigarettes, would provide important insights.

Based on our findings, it may be worthwhile health care professionals asking patients who have quit smoking about their use of e-cigarettes. They could discourage uptake of vaping among long-term ex-smokers who have not used e-cigarettes and advise those who vape to reduce or quit if there is little risk of relapse to smoking. There is an emerging literature on vaping cessation that may be useful to draw upon in these interactions [31].

Strengths of this study include the representative sample, monthly data collection, and comprehensive assessment of vaping behaviour. There were also limitations. Information was not collected on the timing of vaping uptake (or re-uptake after discontinuation) or the use of e-cigarettes in quit attempts that occurred more than a year ago, so our definitions of early and late uptake of vaping after smoking cessation were limited by the information we had available. As a result, the subgroups we analysed were not exhaustive. For example, our definition of recent uptake only included past-year quitters and therefore excluded people who took up vaping just over a year after quitting smoking. Likewise, our definition of late uptake was linked to a specific calendar year and excluded people who quit smoking after 2011 without using e-cigarettes who took up vaping some years later. In addition, data were self-reported and recall may have been imperfect, particularly for events that happened a long time ago (e.g., how long ago the participant quit smoking). However, we would not expect memory failure to differ between vapers and non-vapers. Finally, while the sample was large overall, small sample sizes for certain subgroups (e.g., recent ex-smokers) limited the precision of some estimates. Findings cannot be presumed to generalise to other countries.

Vaping prevalence increased substantially among adult ex-smokers in England over the past decade, particularly at younger ages. While this is likely to have been partly driven by increases in people using e-cigarettes as a smoking cessation aid and continuing to vape beyond their successful quit attempt, there was also evidence of increased uptake of vaping among those who had been abstinent from smoking for many years.

Data are available on Open Science Framework (https://osf.io/87tkw/) with age provided in bands to preserve anonymity.

AUDIT-C:

Alcohol Use Disorders Identification Test – Consumption

1.      Lindson N, Butler AR, McRobbie H, Bullen C, Hajek P, Begh R, et al. Electronic cigarettes for smoking cessation. Cochrane Database Syst Rev. 2024. https://doi.org/10.1002/14651858.CD010216.pub8.

Article PubMed Google Scholar 

2.     Jackson SE, Brown J, Beard E. Associations of prevalence of E-cigarette use with quit attempts, quit success, use of smoking cessation medication, and the overall quit rate among smokers in England: a time-series analysis of population trends 2007–2022. Nicotine Tob Res. 2024;26(7):826–34.

3.     Zhu S-H, Zhuang Y-L, Wong S, Cummins SE, Tedeschi GJ. E-cigarette use and associated changes in population smoking cessation: evidence from US current population surveys. BMJ. 2017;358:j3262.

Article PubMed PubMed Central Google Scholar 

4.     Wagener TL, Floyd EL, Stepanov I, Driskill LM, Frank SG, Meier E, et al. Have combustible cigarettes met their match? The nicotine delivery profiles and harmful constituent exposures of second-generation and third-generation electronic cigarette users. Tob Control. 2017;26:e23–8.

5.      Article PubMed Google Scholar