Automated E-mail Messaging as a Tool for Improving Quit Rates in an Internet Smoking Cessation Intervention (original) (raw)

Abstract

Objective: The aim of this study was to determine whether an automated e-mail messaging system that sent individually timed educational messages (ITEMs) increased the effectiveness of an Internet smoking cessation intervention.

Design: Using two consecutive series of participants, the authors compared two Web-based self-help style smoking cessation interventions: a single-point-in-time educational intervention and an enhanced intervention that also sent ITEMs timed to participants' quit efforts. Outcomes were compared in 199 participants receiving the one-time intervention and 286 receiving ITEMs.

Measurements: Demographic factors, number of cigarettes smoked, nicotine addiction, depressive symptoms, and confidence in ability to quit were measured at entry. Twenty-four–hour quit attempts and seven-day point-prevalence of abstinence (nonrespondents assumed to smoke) were measured 30 days after each subject's self-selected quit date.

Results: The one-time and ITEMs groups differed in some demographics and some relapse risk factors but not in factors associated with 30-day quit rates. ITEMs appeared to increase the rate at which individuals set quit dates (97% vs. 91%, p = 0.005) and, among the respondents to follow-up questionnaires (n = 145), the rate of reported 24-hour quit efforts (83% vs. 54%, p = 0.001). The 30-day intent-to-treat quit rates were higher in the ITEMs group: 7.5% vs. 13.6%, p = 0.035. In multivariate analyses controlling for differences between groups, receiving ITEMs was associated with an increase in the odds ratio for quitting of 2.6 (95% confidence interval = 1.3–5.3).

Conclusion: ITEMs sent on strategic days in smokers' quit efforts enhanced early success with smoking cessation relative to a single-point-in-time Web intervention. The effect appears to be mediated by ITEMs' causing smokers to plan and undertake quit efforts more frequently.

The objective of this study was to evaluate the effectiveness of automatically generated e-mail messages in improving quit rates in an online smoking cessation study. There is a large body of literature showing that computer-based interventions for smoking cessation can have substantial efficacy.1–3 It is not surprising, then, that as computing has migrated to the Web, so have computer applications for smoking cessation. Web-based smoking cessation help is among the most widely available services on the Web. A growing number of sites offer help on a fee-for-service basis.4 However, almost no work has been published to date on the effectiveness of different approaches to smoking cessation on the Internet. This study seeks to add to the body of knowledge on the effectiveness of different Internet-based strategies for smoking cessation and behavioral change.

The Internet is a diverse medium offering several channels for communication to foster behavioral change, such as smoking cessation. Work on the use of this medium for behavioral change, in general, is in the early stages, and little is known about how best to use the different communication channels afforded by the medium.5 In addition to the more traditional static delivery of educational material on Web pages, the Web affords opportunities to implement interventions with tailored educational materials and to use automated approaches to assist participants with planning.

E-mail is one of the most popular channels for medical activity on the Web. Most activity is focused on support groups. These online communities combine bulletin boards and electronic mail to create virtual communities. There are no published studies at this time demonstrating the efficacy of such communities in aiding smoking cessation, although they are quite popular. Internal studies by sites such as QuitNet report quit rates of around 14% over the short term.4 E-mail is also used for communication between smoking cessation counselors and clients in some commercial sites. The effectiveness of this approach in enhancing quit rates is not known; however, it has been shown to be an effective approach for enhancing weight loss.6

Given the diverse mix of options for communication and several competing theories for inducing behavioral change,7,8 there is a need for data to inform the construction of interventions. The objective of this study was to understand the impact of different components on the outcomes of Internet cessation efforts. Our focus was on the use of automated e-mail messaging technologies in enhancing the effectiveness of static educational materials on the Web. Automated e-mail applications combine databases of participant information and educational materials. These systems use software-encoded rules to determine which educational materials should be delivered to which person at a given time. The materials are then automatically sent out via e-mail. These systems differ from traditional Web sites in that content is delivered to participants at individually determined strategic times rather than waiting for participants to return to a Web site for more information. These applications can be combined with Web sites. If the e-mail message has a uniform resource locator (URL) that is embedded in the message, then this can bring back participants to the Web site with a click of the computer mouse. If coded identifying data for the subject are included in the URL, the site can recognize the participant and be highly tailored to that subject upon return. These techniques are in wide use by Web merchants and other organizations to assist with commercial activities and communications. In this study, we sought to assess the effectiveness of such methods in smoking cessation.

Methods

This study used two consecutive series of participants to assess whether the addition of an automated e-mail messaging system that delivered individually timed educational messages (ITEMs) at strategic intervals improved early outcomes in smoking cessation. We compared outcome data from two Web sites of similar designs: one with extension of the other's capabilities—an education-only Web site and an education-plus-ITEMs Web site. We recruited participants and obtained outcome data using similar methods in both series. Data collection periods were separated by between six and 14 months.

The education-only Web site is described in detail in Stoddard et al.9 The site offered an intervention with tailored and untailored educational materials. After digitally signing informed consent, participants completed an extensive online questionnaire. As they completed each page, they received tailored feedback on the implication of their responses for their quit efforts. The questionnaire collected data on smoking history, depressive symptoms,10,11 level of nicotine addiction, menstrual symptoms (women only), and motivation for quitting. After completing the full questionnaire, participants received a summary of the site's recommendations for their quit effort.

They then were taken through static online educational materials (which they could return to at any time or download) with motivational information on how to get ready to quit, how to deal with symptoms from nicotine withdrawal, how to avoid tempting situations, what to do if they should relapse, and how to help others to quit after they were successful. This content was adapted from a brochure that had been used widely in prior smoking cessation research studies as the educational control condition.12–14 The materials were based loosely on the stages of change model of Prochaska and DiClemente15 and Fischbein's16 theory of reasoned action. After reviewing these materials, participants were presented with a calendar and asked to choose a quit date within the next 30 days. This ended the intervention. This Web site was operational between May 2000 and April 2002 at the URL http://stopsmoking.ucsf.edu.

Recognizing the limits of a one-time intervention education-only site or control for smoking cessation, we began development of the enhanced site, a Web site that used e-mail to provide timed delivery of educational materials to trial participants. The prior control site's educational questionnaire, software for generating advice on quit efforts from the questionnaire, educational materials, and software for selecting quit dates were carried over unchanged into the new intervention site. In addition, the new intervention offered participants a few tools to help them with their quit effort over time. These included tools for tracking their cigarette use over time and then displaying the reported data on a graph, an online private journal, and an online checklist for the week before their quit effort to help participants prepare for their quit day. Users could access these features by logging into the site with their user name and password. However, the primary enhancement was to use ITEMs to assist participants with quit efforts. Once participants had set a quit date, the site sent e-mail messages to participants with content particularly relevant to the stage of their quit effort. Each message contained a few brief words of encouragement and a URL that, when clicked by the e-mail clients, linked to a Web browser, took the participants back to the site, logged them into their account, and brought up educational materials staged to the point in the participants' quit effort. For example, at seven days prior to the quit date, they were brought to a Web page with information about how to get ready to quit and a reminder to use the pre-quit date checklist on the Web site. Specific e-mails sent to participants are detailed in Table 1. This site was operational at the URL http://stopsmoking.ucsf.edu from May 2002 to November 2003.

Table 1

E-mail Messages Sent as Part of the Intervention

Table 1

E-mail Messages Sent as Part of the Intervention

Web Site Design and Implementation

Web sites were implemented in HTML and Active Server Page scripts. Web sites tracked each session by means of a cookie and encrypted all traffic using secure socket layer protocol. Participants' data were stored in a Microsoft Access database running on a separate server. Access to the Web survey was limited to the http protocol by multiple firewalls.

Mailer Design and Operation

The Web site's software automatically generated e-mail messages. The mailer system consists of two components: the mailer database and the mailer script written in Visual Basic. The mailer database contained three tables: a table of queries defining eligibility for e-mail messages (the Query Table), a table defining e-mails to be sent and containing elements for personalization (the Message Table), and a table tracking e-mail messages previously sent (the Index Table). The mailer script was executed daily. It first identified all instances of a particular e-mail message that might need to be sent by running queries against the study's main database. For each participant in the study, it returned a record set from the Query Table, and each record in the retrieve set was assigned a unique query index. Next, the software checked this against the Index Table to see whether this particular e-mail already had been sent to a user. If a query index was found in the table, the site checked how many times a message had been sent and determined whether a message has been sent more than the maximum number of specified times. If not, a third query was made to the Message Table, which generated a personalized and/or tailored e-mail message that was then sent out to the participant. The event of sending the e-mail message was recorded in the Index Table.

Outcome Assessment

To assess the outcome of use of the site, participants enrolling in the study were sent e-mail messages 30, 90, 180, and 360 days after their quit date or, if a quit date had not been set, the same number of days after enrollment. Participants who did not respond received up to three repeat requests separated by two days, with messages that asked participants to return to the site and to complete a brief questionnaire on the outcome of their quit effort. The questionnaire collected data on three basic outcomes:

1. Had participants had a 24-hour smoke-free period in the past 30 days (a 24-hour smoke-free period being an accepted indicator of a serious quit effort)?
2. Had participants smoked at all in the past seven days (a seven-day abstinence being a common measure of a successful smoking cessation)?
3. If they had smoked, how many cigarettes had they smoked, each day on average?

After collecting these basic outcome data, the site also presented participants with a variety of questions exploring their mood [Center for Epidemiological Studies-Depression Scale (CES-D)10 and major depressive episodes screener11] and other symptoms they may have experienced during their quit efforts. No incentives were provided for response to questionnaires.

Population

Participants were recruited to the Web site via search engine activity and Web browsing. To have as comparable a population as possible for the comparison, we selected the last group of participants enrolling in the education-only intervention. These were participants who enrolled in the study between February 1, 2002, and April 1, 2002, and had completed the online questionnaires. The ITEMs group was composed of individuals recruited between December 1, 2002, and November 12, 2003, who also had completed online questionnaires.

Data Analysis

Analytic files were assembled from Web databases for each study. We computed descriptive statistics on each group and compared the two groups using Pearson χ2 statistics for categorical variables or ordinary t-tests for continuous variables. We then compared groups on process measures: setting quit dates, response rates to requests for outcome data, and reported 24-hour quit attempt rates. Finally, we compared the two groups with regard to the proportion of responses to requests for outcome data and on actual outcomes (seven-day abstinence). Analyses of outcomes assumed that persons not responding to e-mail requests continued to smoke.

Multivariate methods were used to attempt to control for differences between subject groups in the study. To determine which demographic factors and smoking-related factors, if any, were related to a successful seven-day abstinence, we first performed one-way analyses using logistic regression methods. Then using a stepwise regression, with entry criteria to the model of 0.2 and existing criteria of 0.1, we determined the best predictive model based on demographic and other variables. To this “best” model, we added an indicator variable for the intervention received by the subject. We then computed the odds ratio for the effect of the intervention. Calculations were performed initially using ordinary least squares methods.

Results

The analytic data set for this study had a total of 485 study enrollees. Of these, 199 received the education-only intervention and 286 the ITEMs intervention. Details on the demography of the two populations are presented in Table 2. Subjects receiving the ITEMs intervention had a higher educational level and a higher rate of full-time employment. Regarding the smoking history of the two groups, subjects in the e-mail intervention smoked slightly fewer cigarettes on average (20.4 vs. 22.1, p = 0.01) but were more addicted to nicotine as measured by the Fagerström scale for Nicotine Dependence (5.7 vs. 5.3, p = 0.034). In terms of process measures, the ITEMs group had a higher rate of setting a specific quit date (97% vs. 91%, p = 0.005). Subsequent analyses rely on participants responding to e-mail requests to return to the site and provide follow-up data. Response rates were relatively low (35% in the comparison group and 26% in the ITEMs group, respectively, p = 0.034). Among respondents, important differences were seen. Eighty-three percent of respondents in the ITEMs group engaged in a quit effort, whereas only 54% of respondents in the education group undertook such an effort (p = 0.0002). Testing for a difference between the groups in the proportion of individuals reporting seven-day abstinence, we found significant differences, with 7.5% of persons receiving educational content alone quitting and 13.6% of persons receiving the ITEMs intervention quitting (p = 0.035).

Table 2

Differences between Experimental Groups and the Univariate Associations between These Factors and Outcomes at 30 Days Post-quit Date

Results show that although there are differences between the enrollees in experimental groups, they are in factors that are not strongly associated with outcomes.

Table 2

Differences between Experimental Groups and the Univariate Associations between These Factors and Outcomes at 30 Days Post-quit Date

Results show that although there are differences between the enrollees in experimental groups, they are in factors that are not strongly associated with outcomes.

Because of the differences in the demographic and smoking history–related factors between groups, we examined one-way associations between these factors and the primary outcome of self-reported abstinence for seven days. Analysis assumed that persons not responding to questionnaires continued to smoke. The results are also shown in Table 2. Factors that were associated with a higher probability of a participant's having quit smoking were the participant's confidence in his or her ability to quit (self-efficacy) and having a low score on the CES-D (having fewer symptoms of depression). Groups were balanced with regard to these factors. No other significant associations were found.

To control for potential differences between groups, we created a multivariate model with factors that differed between groups or were associated with successful quitting in univariate studies. To facilitate interpretation of multivariate results, we report results with continuous variables discretized into binary outcomes based on the median score (but performed analyses with continuous methods first to ensure stability of results). To this model, we added an indicator variable for the intervention that the patient received. Regression results are shown in Table 3. Results suggest that, after controlling for differences in demographics and risk factors for relapse, the ITEMs improved the odds of a successful quit effort by approximately 2.5 times (odds ratio, 2.66; 95% confidence interval = 1.3–5.4).

Table 3

Multivariate Regression Analysis of Variables That Differed between Groups or Were Associated with 7-Day Abstinence at 30 Days

SE = standard error; Prob = probability.

Table 3

Multivariate Regression Analysis of Variables That Differed between Groups or Were Associated with 7-Day Abstinence at 30 Days

SE = standard error; Prob = probability.

Discussion

In this study, we showed that enhancement of a smoking cessation Web site with ITEMs, sent via e-mail at strategic times in smokers' quit efforts, increased the rate of early successful quit efforts. There are a number of caveats to this result. The result is based on a historical comparison group and the characteristics of persons visiting the site changed somewhat over the period of data collection, presumably due to changes in how the site was listed with search engines, advertising, etc. Participants were similar in their level of addiction to nicotine, number of cigarettes smoked (though these variables show statistical differences due to the large sample size), duration of smoking, level of depression, and confidence in quitting. Adjusting for these differences using a multivariate statistical model appeared to strengthen differences in outcomes between groups.

One important limitation of this study is that the ITEMs intervention included more components than just reinforcing e-mail messages. The intervention also included a journal and an online tracker for cigarette use. These components could also have contributed to the observed results. These components were added to keep our site competitive with the features of other sites offering smoking cessation services on the Web. Further studies are needed, with larger sample sizes, to elucidate the specific mechanisms of effects, although we believe that the effects of these components were small.

The result is seen at a relatively early period in the process of becoming a nonsmoker. Our prior data from Web studies of smokers9 suggest that this is the maximum time of response and the best time to detect an effect of an intervention on the process. However, these data do not show that e-mail improves long-term quit rates. Rather, the data argue only that it helps individuals undertake more attempts and achieve higher rates of early success. At this stage in the development and evaluation of Web-based smoking cessation interventions, a focus on short-term effects seems appropriate. As in traditional face-to-face interventions, once we have found the most effective short-term interventions, we will need to develop and study interventions that have long-term effects.

The result of this study is based on the use of e-mail and Web technology to elicit outcome data. Previous studies have shown that a substantial proportion of participants in online studies will respond to e-mail requests for follow-up information on quit efforts and complete online questionnaires. For example, in our initial study,9 approximately 42% of participants were willing to respond to e-mail requests for this information. By linking requests to incentives for response, Feil and colleagues17 were able to achieve a 46% follow-up rate in long-term users of a Web site. This approach introduces its own types of biases, including a social acceptability bias. Ultimately, the developers of Web applications for smoking cessation and behavior change need to know whether their applications, in their native contexts without incentives, have effects on behavior. For this reason, our study did not use incentives, and as a result, our response rates were relatively low, especially with the ITEMs intervention group.

Among respondents, rates of seven-day abstinence were relatively high. However, it is unlikely that nonrespondents quit in the same proportion as respondents. We therefore conservatively assumed that nonrespondents continued to smoke—a common assumption in analyses of smoking studies. This produced one-month quit rates of approximately 13.5% in the ITEMs group. The odds ratio for quitting at 30 days was 2.55. This compares favorably with most self-help interventions for smoking cessation. Self-help interventions typically have weak effects—odds ratios that average approximately 1.3 compared with no help.18 The increment in effectiveness observed over the self-help comparison group was large compared with the effort required. For example, a review of group behavior programs found that adding these labor- and time-intensive programs to a self-help intervention increased the odds of quitting by only approximately 1.9.19

The ITEMs intervention Web site is fully computerized and requires no more attention than any other Web site. Therefore, even if our smoking Web site was ultimately effective in only a small percentage of persons who use the site, it could still have a relatively large public health impact because of the low unit cost of delivery of this intervention.20

Two previous studies have shown in randomized trials that e-mail communications can enhance the effectiveness of other medical interventions; however, these studies focused on e-mail messages generated by people. Lorig et al.21 demonstrated in a randomized trial that a closed, moderated e-mail discussion group increased the efficacy of a self-help program with a textbook and other materials in the treatment of chronic back pain. This type of activity requires extensive effort on the part of the leader of the group to filter and respond to messages. Tate and colleagues6 have examined the use of e-mail counseling to enhance a weight loss program. This program also required intensive individual efforts from counselors. Similar to Lorig et al., these investigators found significant enhancements in weight loss when counselors provided individualized feedback via e-mail. Of course, the use of counselors or moderators increases the costs of delivery of the intervention.

This is the first reported study, to our knowledge, of the effects of automated e-mail messaging on quit rates. Etter et al.22 examined the effect of automated e-mail messaging on smokers' motivation to quit. The research group sent e-mail messages to registrants at a Web site for persons interested in smoking cessation. Respondents (25% response rate) indicated that messages encouraging the use of the nicotine replacement to cut down on the number of cigarettes smoked per day increased smokers' intention to quit. Two other messages that advised recipients of the adverse effects of combining nicotine replacement with smoking or urged the use of nicotine replacement in situations in which smoking was forbidden did not have effects of stated intention to quit.

Conclusion

An automated e-mail system that delivered ITEMs strategically timed to participants' quit efforts more than doubled the early effectiveness of an online self-help smoking cessation intervention. The effect appears to be mediated through the messaging system's getting more smokers to set quit dates and undertake quit efforts. Automated e-mail systems that deliver ITEMs at strategic times are a potentially important component of Internet software systems to promote behavioral change.

References

1

.

Evaluating a population-based recruitment approach and a stage-based expert system intervention for smoking cessation

.

Addict Behav

2001

;

26

:

583

–

602

.

2

.

The efficacy of computer-tailored smoking cessation material as a supplement to nicotine polacrilex gum therapy

.

Arch Intern Med

2000

;

160

:

1675

–

81

.

3

.

Effectiveness of a computer-tailored smoking cessation program: a randomized trial

.

Arch Intern Med

2001

;

161

:

2596

–

601

.

5

.

Do Internet interventions for consumers cause more harm than good? A systematic review

.

Health Expect

2002

;

5

(

1

):

28

–

37

.

6

.

Effects of Internet behavioral counseling on weight loss in adults at risk for type 2 diabetes: a randomized trial

.

JAMA

2003

;

289

:

1833

–

6

.

7

.

Beyond stages of change: the quitting continuum measures progress towards successful smoking cessation

.

Addiction

1998

;

93

:

277

–

86

.

8

et al. .

Systematic review of the effectiveness of stage based interventions to promote smoking cessation

.

BMJ

2003

;

326

:

1175

–

7

.

9

.

Smoking cessation research via the Internet: a feasibility study

.

J Health Commun

2005

;

10

(

1

) in press.

10

.

The CES-D Scale: a self-reported depression scale for research in the general population

.

Appl Psychol Meas

1977

;

1

:

385

–

401

.

11

.

Preventing major depression by promotion of emotional regulation: a conceptual framework and some practical tools

.

Int J Ment Health Promot

1998

;

1

:

23

–

40

.

12

.

A comprehensive smoking cessation program for the San Francisco Bay Area Latino community: Programa Latino para Dejar de Fumar

.

Am J Health Promot

1993

;

7

:

430

–

42

, 475.

13

.

Evaluation of “Guia para Dejar de Fumar, ” a self-help guide in Spanish to quit smoking

.

Public Health Rep

1991

;

106

:

564

–

70

.

14

.

Mood management mail intervention increases abstinence rates for Spanish-speaking Latino smokers

.

Am J Commun Psychol

1997

;

25

:

325

–

43

.

15

.

The Transtheoretical Approach: Crossing Traditional Boundaries of Therapy

.

Malabar, FL

:

Krieger Publications

,

1994

.

16

.

Developing effective behavior change interventions: some lessons learned from behavioral research

.

NIDA Res Monogr

1995

;

155

:

246

–

61

.

17

.

Evaluation of an Internet-based smoking cessation program: lessons learned from a pilot study

.

Nicotine Tob Res

2003

;

5

:

189

–

94

.

18

.

Self-help interventions for smoking cessation

.

Cochrane Database Syst Rev

2002

(

3

):

CD001118

.

19

.

Group behaviour therapy programmes for smoking cessation

.

Cochrane Database Syst Rev

2002

(

3

):

CD001007

.

20

.

Using new information technology to treat tobacco dependence

.

Respiration

2002

;

69

(

2

):

111

–

4

.

21

.

Can a back pain E-mail discussion group improve health status and lower health care costs? A randomized study

.

Arch Intern Med

2002

;

162

:

792

–

6

.

22

.

Impact of messages on concomitant use of nicotine replacement therapy and cigarettes: a randomized trial on the Internet

.

Addiction

2003

;

98

:

941

–

50

.

Supported by grants 7RT-0057 and 10RT-0326 from the Tobacco Related Disease Research Program of the University of California (Ricardo F. Muñoz, Primary Investigator). Dr. Muñoz is supported through the University of California, San Francisco/San Francisco General Hospital Latino Mental Health Research Program, which is funded by the University of California Committee on Latino Research. The authors thank the members of the programming and development team, including Aditya Bansod, Dawna Perkins, and Jacquie Parry, who helped design the e-mail messaging system, and members of the team that developed the online educational content, including Eliseo Pérez-Stable, Jacqueline Stoddard, and Carlos Penilla.

American Medical Informatics Association