Commentary: Pediatric eHealth Interventions: Common Challenges During Development, Implementation, and Dissemination (original) (raw)

Abstract

Objective To provide an overview of common challenges that pediatric eHealth researchers may encounter when planning, developing, testing, and disseminating eHealth interventions along with proposed solutions for addressing these challenges. Methods The article draws on the existing eHealth literature and the authors’ collective experience in pediatric eHealth research. Results and conclusions The challenges associated with eHealth interventions and their proposed solutions are multifaceted and cut across a number of areas from eHealth program development through dissemination. Collaboration with a range of individuals (e.g., multidisciplinary colleagues, commercial entities, primary stakeholders) is the key to eHealth intervention success. To ensure adequate resources for design, development, and planning for sustainability, a number of public and private sources of funding are available. A study design that addresses ethical concerns and security issues is critical to ensure scientific integrity and intervention dissemination. Table I summarizes key issues to consider during eHealth intervention development, testing, and dissemination.

Keywords: design, eHealth, implementation, mHealth

Over the past two decades, eHealth interventions have rapidly grown in popularity (Fatehi & Wootton, 2012). eHealth has been defined in numerous ways ranging from “using the Internet and other electronic channels for the access and delivery of health and lifestyle information and services” to “health promotion delivered and managed over the Internet” (see Table 3 in Oh, Rizo, Enkin, & Jadad, 2005). For the purposes of this Commentary and to be consistent with prior literature within health psychology, eHealth is defined as the use of technology to “function as an active ingredient in treatments” such as health behavior interventions (Cushing & Steele, 2010, p. 937). This includes using technology to gather information from patients and to provide support and guidance to patients and their families to promote improved health outcomes (Cushing & Steele, 2010; Palermo, 2008). Examples of eHealth technologies include Internet Web sites, mobile phone-enabled capabilities (e.g., text messaging, software applications), computer games, CD-ROMs, tablets, and computers.

Among pediatric populations, a growing number of eHealth interventions has been tested and made available (Cushing & Steele, 2010; Gustafson et al., 2012; Palermo, Wilson, Peters, Lewandowski, & Somhegyi, 2009; Ritterband et al., 2013; Stinson, Wilson, Gill, Tamda, & Holt, 2009). Children and adolescents appear particularly amenable to using eHealth programs because they may have greater facility and comfort with and spend more time using technologies such as the Internet and mobile phones (Bennett, Maton, & Kervin, 2008). Although there is a growing literature on the effectiveness of eHealth interventions among pediatric populations, considerably less attention has been paid to the logistical issues associated with developing, testing, and disseminating these interventions. Although there has been some discussion of these topics in the adult literature (Ahern, Patrick, Phalen, & Neiley, 2006; Danaher & Seeley, 2009; Eng, 2002; Pagliari, 2007), eHealth interventions being delivered to children and families have their own distinct challenges and considerations. Further, although eHealth interventions can be unique in their design and purpose, there are common issues that arise when developing and implementing these interventions in the pediatric population. These design, planning, ethical, and logistical issues are particularly important to examine because they have the potential to significantly hinder or facilitate the eHealth research. Thus, the goals of this commentary are to review key challenges that pediatric eHealth researchers may encounter when planning, developing, testing, and disseminating new eHealth interventions and to propose possible solutions for addressing these challenges based on the literature and the authors’ experiences conducting pediatric eHealth research. An overview of key issues related to eHealth intervention development, testing, and dissemination can be found in Table I.

Table I.

Checklist of Tasks and Considerations for eHealth Program Projectsa

Planning for eHealth Program Creation

Assembling a Development Team

eHealth applications are typically created by interprofessional teams (Atkinson & Gold, 2002; Ritterband et al., 2003; Schueller, Munoz, & Mohr, 2013), consistent with current conceptualizations of team science (Sellers, Caporaso, Lapidus, Petersen, & Trent, 2006). Although the exact team members differ based on the needs of a particular project, team members frequently include a content area specialist (e.g., pediatric psychologist focused on a particular illness population) and technology developers (e.g., programmer; Table I). Depending on the scope of the eHealth program, other stakeholders and collaborators who provide input may include patients and their families, health-care providers (Riley, Glasgow, Etheredge, & Abernethy, 2013), information technology vendors, e-learning and human factor consultants, policymakers, insurers, third party payers, health economists, health-care administrators, community leaders, epidemiologists, instructional designers, and business and marketing professionals (Pagliari, 2007).

Clear and effective communication among all team members (Table I), including the target audience, is of utmost importance to the development of eHealth programs (Pagliari, 2007). For example, before the development of an electronic tutorial for school nurses (Steele, Wu, Cushing, & Jensen, 2012), the research team conducted formal focus groups to determine the perceived needs for weight-related health communications (i.e., content), preferred instructional session length, preferred means (or process) of communication (e.g., videos, slide shows, text), and preferred methods of learning assessment (e.g., quizzes). These preferences were taken into account when developing the content and structure of the tutorial (Table I).

It is advantageous for team members focused on development or research to share a common understanding of both the therapeutic elements of the intervention and the design elements of the program or application. The development team will be more likely to design successful digital components in eHealth programs if they have a basic understanding of the goals of the intervention and the therapeutic processes that are necessary for a successful outcome. Likewise, researchers and clinicians that acquire at least a rudimentary understanding of the limits and capabilities of the technologies may be better able to conceptualize how therapeutic components can be rendered in eHealth platforms.

Cost Considerations

One of the most often underappreciated and underestimated aspects of conducting eHealth research is accurately estimating the cost of developing and testing the program (Table I). Although costs range widely and depend on many factors, such as the skill level of the team members, complexity of the desired program or application, and funding and timeline restrictions, rarely is eHealth intervention development inexpensive. In addition to technology developer costs, researchers should consider the costs of other team members (e.g., compensation for time; Atkinson & Gold, 2002). Some members, such as a team leader (i.e., PI), research coordinator, or content area consultant may devote specific effort and consequently receive a set payment schedule, regardless of the actual time required to develop the eHealth program. Other team members, such as external programmers, may require different billing structures (e.g., payment in lump sum or hourly) that are contingent upon actual time working on the eHealth program.

When hiring programmers or technology developers, there are potential benefits and challenges to hiring developers within one’s institution (e.g., information technology office within a university) or external to one’s institution (e.g., commercial Web site design group). Internal programmers may be of lower cost, are more easily accessible than commercial consultants, and are familiar with institutional technology policies. However, there is great variability in costs, expertise, and experience across technology developers both within academic- or hospital-based institutions and those external to these institutions. For example, some internal collaborators may not have prior experience with the programming required for the eHealth program or with creating databases for research that output participant’s data. Finding a development team that has prior experience with eHealth applications may significantly improve the overall experience due to improved efficiency and better mutual understanding of the goals. Table I contains recommendations for eHealth program budget items.

eHealth program development can often be delayed because of unforeseen technological challenges. It may prove prudent to budget for extra program development time to account for these technological challenges. For instance, if an eHealth program is estimated to take 9 months to develop, a researcher may want to allocate 12 months for development. Costs can also be influenced by the complexity of the desired eHealth program. As a result, it can be helpful to think of eHealth program development as an iterative process in which the higher priority components are developed first (e.g., those that have the most supporting evidence) and lower priority components are added in future versions (Table I; Ritterband, Thorndike, Cox, Kovatchev, & Gonder-Frederick, 2009).

Obtaining Funding

A number of venues offer possibilities for funding eHealth program development and testing, including institutional resources, foundations and private investments, and public research support. Increasingly, universities and other academic institutions are developing centers for technology commercialization and transfer that may provide seed money for pilot projects. For example, the University of Kansas and the University of Kansas Medical Center have recently developed the Center of Technology and Commercialization—an affiliated 501(c)(3) non-profit organization with the specific mission of “maximizing the impact of university intellectual property” through commercialization of inventions created at the university, generating revenue to support research, facilitating collaborative research projects, and assisting with legal and financial matters related to program development (e.g., patent and licensing; www.ctc.ku.edu). Similar entities exist on numerous campuses (University of Oklahoma, http://otd.ou.edu/about/; University of Virginia, http://innovation.virginia.edu/; Cincinnati Children’s Hospital Medical Center, http://www.cincinnatichildrens.org/research/cincinnati/support/ctc/default/) and can be of benefit to researchers interested in developing or commercializing a specific eHealth program. Typically, institution-based resources such as the examples mentioned here are available to individuals affiliated with the institution. Researchers who do not have similar offices available to them may be able to partner with existing groups or individuals who have expertise in commercialization, such as through institutional offices for legal services.

Research and philanthropic foundations and private investors may also supply material or instrumental support for development of eHealth interventions consistent with their specific missions. For instance, foundations frequently have targeted research or clinical missions; eHealth applications that address specific components of a foundation’s mission will likely have more success than applications that address issues that are not directly germane to the foundation’s mission. As an example, two of the authors (R.G.S., Y.P.W.) developed an electronic tutorial to enhance school nurses’ communications with families about weight-related health (Steele et al., 2012) with funding from the Health Care Foundation of Greater Kansas City. The Foundation awarded the funds because of the correspondence between the specific eHealth intervention and their own mission (i.e., to eliminate barriers and promote quality health for uninsured and underserved in [their] service area). Similarly, another of the authors (M.C.) received foundation (e.g., National Headache Foundation) and private funds (e.g., AstraZeneca) for the development of eHealth interventions within the respective organizations’ mission areas.

Public research funding (e.g., federal grants) also is commonly used to develop and research innovative eHealth applications. Traditional funding mechanisms (e.g., R01, R21) remain viable, albeit increasingly scarce, options. The more exploratory of these mechanisms (R21, R34) may be better suited for the initial development of eHealth interventions. However, they are also considerably smaller funding streams and more time limited than the R01 mechanism. Among the authors, there are multiple examples of eHealth interventions developed using federal R-level grants in the area of pediatric chronic pain (T.M.P.), encopresis (L.R.), and arthritis (M.C.). The Small Business Innovation Research (SBIR) and Small Business Technology Transfer (STTR) mechanisms are another possible pathway of federal funding options. These mechanisms encourage partnerships between researchers and small businesses with the goal of developing technological innovations for eventual dissemination to the public (http://www.sbir.gov/). For example, one of the authors (L.R.) has been working with a company (ArchieMD; http://www.archiemd.com) to test an iPad application focused on children with asthma as part of an SBIR grant. The company provides the technical expertise, and the researcher and team conduct a trial for evaluation with the ultimate goal of making the application widely available. Please see Table I for a summary of common funding options.

For new investigators and researchers starting in eHealth who may not yet be competitive for federal funding, several strategies to build an eHealth program while minimizing costs could be used. First, researchers can prioritize building and testing smaller components of the envisioned larger program. For instance, a researcher who wants to build a Web site that provides health education and feedback to patients based on self-monitoring via text messaging prompts could first begin by building a simplified version of the Web site and later adding the text messaging component. Building one component at a time can save funds and time and enables a researcher to demonstrate feasibility and gather initial data to support efforts to obtain additional funding. In addition, researchers can prioritize eHealth components that are most feasible for the researcher to create with minimal assistance or using intramural funds. Alternatively, a researcher can build an initial eHealth program that uses existing technology, such as commercially available programs (Cushing, Jensen, & Steele, 2011). Although using commercially available programs lowers costs and decreases development time, researchers are typically limited in their ability to modify the program to their unique needs.

Developing the eHealth Program

As noted by Cushing and Steele (2010), effective eHealth interventions share a number of commonalities with face-to-face interventions. Chief among these is the need for both in-person and eHealth interventions to be based upon solid empirically based theories of behavior change. Just as the theory of behavior change underlies face-to-face delivery of behavioral intervention, the delivery of treatment solely by or augmented with technology demands the same considerations (Ritterband et al., 2009; Ritterband & Tate, 2009; Table I). Many current technology-based interventions have no theoretical basis (Riley et al., 2011; Webb, Joseph, Yardley, & Michie, 2010). As described by Ritterband et al. (2003), Internet interventions are usually based on effective face-to-face behavioral treatments that have been operationalized and transformed for Internet delivery. However, it could be argued that the transformation from face-to-face to an eHealth format may change the content or intensity of a treatment in ways thought crucial to its success (e.g., removing a live therapist may affect treatment adherence and participation). However, eHealth programs also provide new opportunities not realized with face-to-face delivery (Ritterband et al., 2009, e.g., increased opportunity for assessment, real-time delivery of intervention in situ).

eHealth interventions frequently diverge from face-to-face interventions with regard to access to therapeutic material, immediacy of feedback, and the frequency of prompts for behavior change or assessment of health behavior (Norman et al., 2007). Developers must consider how their eHealth intervention will capitalize on these unique aspects of the modality. On the other hand, a potential limitation of eHealth interventions is the lack of face-to-face interactions, which may facilitate engagement and adherence. It is thus incumbent on the developer to ensure that eHealth interventions contain elements that engage clients/patients to “stand on their own”—without the benefit of face-to-face interactions.

Improvements in technology are making the technical side of eHealth program development more attainable. For example, eHealth program development can be done through the use of existing infrastructures and frameworks (e.g., computer code). The Research Infrastructure Containing E-interventions (RICE), created by the team at the University of Virginia (L.R.) represents one such system. RICE is a Web-based solution for creating and delivering new interventions as well as managing the associated research studies. It contains a field-tested toolset that includes integrated components used to define and manage the unique study workflow, presentation of study content, assessment criteria, and participant progress. RICE can be tailored according to the specifications of each new study and team of investigators. It has been used in many National Institutes of Health (NIH)-funded trials with excellent success and has been licensed to BeHealth Solutions, LLC (www.behealthsolutions.com) through the University of Virginia Patent Foundation and rebranded as BeStudy Manager™. Similar systems include the Center for Behavioral Intervention Technologies at Northwestern University (http://tech.cbits.northwestern.edu/) and an open-source platform by researchers at the University of Southampton (https://www.lifeguideonline.org/). Benefits to using such scalable systems include use of a tested infrastructure and reduced costs and shorter project duration through improved data accuracy, informed decision making through existing real-time data collection processes (e.g., prospective subject data collection, repeated measure components), and progress and utilization monitoring. In addition, such systems typically allow the collection of data from multiple sites using a centralized secured server system and electronically stored informed consent documents and payment records. Investigators using such existing infrastructures and/or who are implementing multisite studies will want to engage information technology representatives from all institutions to ensure that any conflicts in institutional policies about technology are addressed. Regardless of the system used or created, eHealth program development is truly an iterative process that typically includes testing by multiple stakeholders and representatives of the intended audience.

Ethical and Safety Issues

The ethics of research involving the use of information and communication technologies (e.g., ethical decision making in the absence of face-to-face contact with participants, privacy protections) have been discussed since the advent of computer technologies (Barak, 1999). A number of professional organizations and groups (Ess & AoIR Ethics Working Committee, 2002; The British Psychological Society, 2007) drafted preliminary policies on this topic, primarily focused on the adult technology user. Because eHealth research has expanded to pediatric populations, additional ethical considerations are necessary. Children represent a “vulnerable” population requiring additional protections in research, particularly around their understanding of their own research participation (Kodish, 2003). There are not only unique issues in eHealth research that may increase the nature of risks but also unique features that could enhance children’s understanding of research participation and decision making. To our knowledge, there are no published policies (e.g., from professional societies) concerning ethical considerations in pediatric eHealth research. Available policy pertains to nonresearch applications such as marketing Web sites to children under the age of 13 years and parental notice and consent requirements (Children’s Online Privacy Protection Act). Our discussion is restricted here to the use of eHealth in research with children.

Recently, Henderson, Law, Palermo, and Eccleston, (2012) used examples from Internet research in pediatric pain to summarize several key issues concerning ethical best practices in eHealth research in children. They focused on core research concerns including issues of recruitment, informed consent, debriefing, privacy and confidentiality, participant safety, and the delivery of psychological interventions online. For example, they summarize best practice procedures for informed consent where it is not possible to seek face-to-face consent, requiring researchers to seek consent over the phone or by e-mail (Fox, Morris, & Rumsey, 2007).

During the development phase, plans for identifying, assessing, and managing possible ethical concerns should be developed. This entails a review of both core scientific research issues in children (e.g., recruitment, informed consent, debriefing, privacy and confidentiality, participant safety) as well as special considerations related to the proposed eHealth program (e.g., assurance of age and medical condition, sharing of information). Regulatory issues are identified and managed in different ways across institutions. Investigators should always seek local guidance; we recommend that eHealth researchers seek pre-IRB (institutional review board) submission consultation (if available) to review the scope of the study, recruitment and referral procedures, and nature of the intervention and interaction with study participants via the proposed technology interface (Table I). Complicating these regulatory considerations, a variety of issues and concerns may be raised as a result of some IRB’s lack of experience and familiarity with eHealth research. For example, an issue that was raised in the initial review of an application of one of the authors (T.M.P.) to conduct a multicenter randomized controlled trial of an Internet intervention for pediatric chronic pain was that psychotherapeutic interventions were not permitted to be delivered to individuals living in multiple jurisdictions. The IRB determination was that the research team was practicing clinical psychology outside of local jurisdictions where the researchers were licensed to practice. The IRB based this determination on policies that have been developed about psychotherapy in telemedicine. In this case, it was necessary to expand the IRB’s understanding of eHealth and to educate the IRB about the difference between telemedicine and Internet interventions for research. Use of specific language to contrast telemedicine (e.g., real time, purpose of diagnosis, and treatment planning) and Internet interventions (e.g., asynchronous communication, purpose to test a behavioral intervention of coping skills) helped make the distinction. Ultimately, this issue was resolved but at the expense of delays in the project start.

The pediatric eHealth researcher will also want to consider whether there are unique aspects of their planned study design and protocol implementation that may require special ethical considerations. For example, researchers working on interventions for common or relatively normative conditions (e.g., eating behaviors) may be interested in enrolling patients directly from the community. In such a scenario, the screening and enrollment of patients may occur through self-referral, without a gatekeeper (e.g., without involvement of a physician or care provider). This is in contrast to recruitment approaches that require physician referral and where clinician participation is solicited (Ritterband et al., 2013). Such differences in approach may have implications for how investigators choose to verify conditions or symptoms reported by the child participant, and whether this can be done effectively online or will require some direct contact.

In addition, emerging technologies present unique or evolving challenges to understanding the potential impact on standards for protection. Although a protocol being delivered at present may appear to have appropriate safeguards for protecting children’s identities, such threats to safety and confidentiality may change over time. For example, types of identifiable information may change over time as children use new and emerging technologies (e.g., use of location services features that provide information on where children are, geographically). These emerging technologies can create challenges in upholding the highest standards for protection over the course of a research project.

Security and Privacy Issues

Security and privacy issues are central to creating and delivering ethical eHealth programs (Campbell, Vasquez, Behnke, & Kinscherff, 2010; Fisher, 2009). Specific safety issues for individual eHealth programs may arise in relation to the type and focus of intervention being delivered (e.g., addressing children’s depressive symptoms), the vulnerability of the population (e.g., adolescents with comorbid pain and depression), and method of follow-up (e.g., automated vs. human support). Investigators should address potential threats to privacy and confidentiality by making a plan for secure hosting (i.e., of the Web site) and taking other precautions that protect privacy and maintain data security (e.g., using screen names rather than actual names in online communications). It may be necessary to separate private health information from other de-identified data by using multiple databases on different servers (Table I).

Moreover, although it is critical to establish a highly secured Health Insurance Portability and Accountability Act (HIPAA)–compliant server infrastructure for intervention delivery, the end users’ environment (e.g., parent and child access at home) must also be considered. eHealth can facilitate collection of data from patients and families throughout the day and in uncontrolled environments where health information could be accessed by others. Thus, developers must consider mechanisms for ensuring that health data are kept private, for example, by requiring users to log in using passwords, having programs automatically close out after a period of inactivity, adding encryption, and placing communications and content behind firewalls (Campbell et al., 2010; Fisher, 2009; Kraut et al., 2004). However, even with these safeguards, users should be made aware of the limits of the security of the technology being used (Manhal-Baugus, 2001). Also, as it is usually the case that online data collection occurs at a remote location (i.e., the participant’s home), researchers and clinicians are advised to consider methods of ensuring that the intended participant is actually the individual completing the measures or intervention. In many ways, these concerns mirror issues that researchers face when sending self-report measures or ‘homework’ home with patients and participants. It may be that this risk is heightened in the eHealth environment. The eHealth researcher is encouraged to consider methods of online security (passwords) and education (instruction) to minimize the impact of unintended users’ input on research results.

Research Issues When Testing the Efficacy or Effectiveness of an eHealth Program

Research Implementation

Several key issues to consider while conducting research on the efficacy and effectiveness of pediatric eHealth interventions include selecting appropriate comparison groups (Danaher & Seeley, 2009; see Table I in current article), addressing technical problems and unique data management issues, monitoring data quality, and adequately assessing for adherence and attrition. Unlike most traditional interventions in pediatric psychology, eHealth interventions depend on the reliability of technology. Technical problems can be common, with consequences ranging from minor participant frustration and project delay to study dropout or lost data. As such, a plan for remedying technical issues during intervention implementation is essential to ensure an adequate empirical test. It can be helpful to create a troubleshooting manual or frequently asked questions document that contains information on addressing anticipated problems such as participants having difficulty downloading software or using technology provided to them (e.g., tablet computer). However, if possible, it can be most useful to hire or include team members who serve as dedicated support person(s) or, at the least, consultants for the duration of the project (Table I).

Studies of eHealth programs often accrue large amounts of data, for example, because of collecting multiple assessments online or tracking extensive information on the images viewed or downloaded by users. Because data accumulate in trials with a large number of participants, the export of large amounts of data on institutional servers for local data analysis can create challenges. Some of these problems can be anticipated in advance, but others may require fixes during the trial with additional programming (e.g., functionality to query smaller data sets for export).

Once the eHealth intervention is ready for testing with participants and patients, it is important to schedule regular monitoring of incoming data. For example, investigators will want to ensure that the format of the data is as expected and that data values appear accurate and within expected ranges. Although these considerations are no less true in standard (i.e., non-eHealth) studies and programs, they have particular relevance in eHealth research where there is rarely a hard copy of assessment measures that could serve as a backup or as a validity check. In our experience, even despite extensive pilot testing, data capture problems can occur during study implementation and need to be addressed on an ongoing basis (Table I). Strategies to improve data accuracy and minimize missing data include having data stored locally (e.g., in a protected folder on a participant’s phone) in addition to having it transmitted wirelessly, using data validation to prevent out-of-range or missing values, and providing backup methods of gathering data in case technology fails (e.g., paper measures).

Adherence and Attrition

There has been considerable discussion in the literature on issues of adherence/attrition in eHealth programs and Internet interventions (Christensen & Mackinnon, 2006; Eysenbach, 2005). It is typically assumed that eHealth trials will incur poor adherence and high attrition. Eysenbach (2005) puts forth a “Law of Attrition” and stated that “… in any eHealth trial a substantial proportion of users drop out before completion or stop using the application …” (p. 1) and that “… high dropout rates may be a natural and typical feature” (p. 1). Although there have been many eHealth trials with poor adherence, it is neither clear nor should it be assumed that this is “typical,” (p. 1) or “a fact of life” (p. 7). Christensen and Mackinnon (2006) suggested that Internet intervention usage may be moderated by a number of constructs, including problem severity (i.e., higher severity associated with higher likelihood to seek professional help), desire for anonymity, limited resources, and variability in preferences for help.

The ongoing debate regarding attrition and adherence is complicated by a lack of clear definitions and standard metrics to quantify them. Christensen, Griffiths, and Farrer (2009) suggested that eHealth adherence is “the extent to which individuals experience the content of the Internet intervention,” and dropout as the term used “to describe an individual who fails to complete the research trial protocol associated with an Internet intervention, and thus does not complete trial assessments.” Christensen and Mackinnon (2006) have suggested that what may appear to be a “dropout,” might actually be an “e-attainer,” or someone who has received what was needed (e.g., experienced symptom reduction) from a program, even if not by the means intended by the researchers or developers (Christensen & Mackinnon, 2006). The most common way to measure adherence is by user log-in. Other metrics include completion of a module or core of content, time, or use of an online tool (Donkin et al., 2011). Although measuring adherence and attrition is important and should be reported for all outcome studies (Table I), examining these constructs across programs and attempting to make generalizations is challenging because of differences in populations, disorders, and diseases.

Dissemination and Sustainability of the eHealth Program

Sustainability of an eHealth intervention refers to its “shelf life” and comprises the technical, financial, scientific, and other infrastructure necessary to maintain the implementation of the eHealth program over time. Sustainability is often overlooked during development or is relegated to an afterthought following an efficacy trial. This may be due, in part, to the field’s commitment to being evidence-based and thus deferring implementation planning until efficacy is established. The unfortunate consequence is that eHealth interventions are frequently abandoned or ignored once a grant project is completed (van Limburg et al., 2011). Widespread adoption of pediatric eHealth interventions will not automatically result from positive results from randomized trials (Curry, 2007; van Limburg et al., 2011). Building on recent calls to create more efficient systems that accelerate the speed of research and thus lead to more sustainable treatments and programs (Riley et al., 2013), sustainability planning should be considered a critical part of pediatric eHealth intervention development to help maintain viability of evidence-based interventions over time (Table I).

A unique sustainability challenge for eHealth interventions is that technologies, how they are used and by whom, are all subject to fairly rapid change—a particular technology medium used to deliver a pediatric eHealth intervention may be obsolete by the point at which the intervention moves to an effectiveness trial or dissemination. Rapid changes in technological “glitz” may result in children becoming bored with and thereby nonadherent to “outdated” programs even if the science behind the intervention is robust. There may also be a “digital divide” within the intended pediatric population such that certain technologies may not be as sustainable in certain segments (e.g., low income or low literacy; Jackson et al., 2008). Technology compatibility can further impede sustainability; eHealth interventions may not be accessible to children that use a different platform (e.g., iPhone vs. Android SmartPhone) or browser (e.g., Safari vs. Internet Explorer) on which the intervention was developed. Initial planning in regards to how children will access the intervention and discussions with the technical support team about their ability to develop the application across platforms can avert subsequent issues or delays during dissemination (Table I).

There are a number of strategies to ensure the sustainability of eHealth programs. At the very least, understanding behavior pertaining to technology adoption (Chiu & Eysenbach, 2010) and how technology is used in a target population helps structure the design of interventions for optimal sustainability. The Pew Internet and American Life Project (http://www.pewinternet.org/) is a valuable resource that provides data on trends in technology use among demographic groups including children and teens. Sustainability can also be maximized by using iterative codevelopment with key stakeholders, as mentioned earlier (Pagliari, 2007; Stinson et al., 2013; van Limburg et al., 2011). Such an iterative process requires modifications to conventional trial methodology and reporting (Eysenbach, 2011). For example, initial stages of eHealth development may be better suited for adaptations of quality improvement methodologies, rather than the traditional model of randomized controlled trials where interventions should not change over the course of the trial. Attention to the cost-effectiveness of eHealth interventions is also particularly important in the current climate of health-care delivery and focus on cost offsets (Tate, Finkelstein, Khavjou, & Gustafson, 2009). Specifically, adoption of eHealth programs on a broader scale will likely be predicated on demonstrations that their efficacy compares favorably with in-person interventions and leads to reduced cost (even after development costs) across the health-care system (Table I).

Dissemination of pediatric eHealth applications can require prudent “business modeling” (Curry, 2007; van Gemert-Pijnen et al., 2011; van Limburg et al., 2011). Building a business case for an eHealth program includes articulating what the intervention is intended to accomplish and the monetary and nonmonetary value of the intervention for all relevant parties (van Linburg et al., 2011). Templates for eHealth business modeling are available and can be a useful resource (e.g., http://www.fp.ucalgary.ca/telehealth/publications/e-Health%20Business%20Case%20template%20v1.4.doc). Options for collaborating with professionals to develop a business case can range from formal collaborations through SBIR/STTR grants, philanthropic collaborations from regional or national business leaders, partnerships with organizations aligned with the population of interest, and collaborations with an institution’s Office of Technology Transfer/Intellectual Property (Table I).

Formalizing collaborations between a commercial entity and an academic institution can help ensure mutual benefit while pre-empting some of the potential conflicting interests pertaining to intervention dissemination. In particular, conflicts can arise based on philosophical differences between academic pursuits and business. Research on a pediatric psychology intervention aspires to be scientifically rigorous (often at the expense of time) and to be transparent about limitations of an intervention; commercial enterprises often maintain a competitive edge through rapid execution of ideas, limiting access to proprietary information, and focusing on return on investment (Eng, 2002). As long as investigator input can be maintained (e.g., as a consultant or board member for the collaborating business entity), leveraging business expertise can be invaluable for optimizing eHealth intervention dissemination. An example of a formal approach to dissemination of eHealth interventions is technology transfer. Technology transfer refers to the process of transferring knowledge and technologies (i.e., inventions, software, or tangible research products) typically from academic institutions to commercial entities to allow for further novel development and dissemination. An institution’s Office of Technology Transfer can be an asset in the dissemination of eHealth interventions by promoting the intervention, gauging commercial interest, identifying potential avenues for licensing out the program to suitable companies, and ensuring researchers receive appropriate credit for the associated intellectual property. As a potential disadvantage, however, investigators may lose influence over what happens to the eHealth intervention if it is licensed to a commercial enterprise. Additionally, the process of technology transfer often requires restrictions on public disclosure about the eHealth intervention (e.g., limiting what information can be presented at conferences) at least until a copyright or patent is filed.

There are additional options for eHealth intervention dissemination beyond collaborations with business and commercialization. Integrating eHealth interventions within existing Health System Information Technology may be viable for certain types of behavior modification interventions that incorporate decision making or other clinical tools for clinicians (Curry, 2007). Direct “marketing” to clinicians at medical conferences or via practitioner organizations can facilitate dissemination to children and families (Table I). Clinician support is often critical for adoption by children and families and is maximized by highlighting the value of the eHealth intervention to patient care and associated savings in time and/or cost (Curry, 2007). However, there are many potential barriers to practitioner adoption of eHealth tools that must first be identified and overcome, such as concerns about privacy for Web-based applications, potential depersonalization of clinical care, and lack of funding for eHealth tool implementation (Anderson, 2007; Curry, 2007).

Conclusions

Although creating and testing pediatric focused eHealth interventions is still a relatively new area of research, there has been substantial work conducted from which to learn and guide future endeavors. Numerous challenges in pediatric eHealth research have been identified and some solutions can now be offered. Models of eHealth program development and testing have been created, and theories of behavior change have been used as program foundations. However, continued testing of eHealth programs consistent with these theories is needed. Many decisions must be made to assemble a strong team, including who to include and how best to weigh costs and benefits. Securing funding is an unremitting issue most researchers struggle to overcome, but diversifying where to seek funds and understanding how to make best use of limited means while keeping larger goals in mind can ultimately result in success. Researchers and developers can also learn from the considerable work that has already occurred in program development and implementation. A number of factors should be considered when designing eHealth programs, including the potential of working on established platforms, prioritizing program features to develop, using proven theory as a basis for the program, ensuring ethical, safety, security, and privacy issues are addressed, and keeping in mind longer term issues of dissemination and sustainability (see Table I for a review of key issues to consider). Rather than “recreate the wheel” each time a new eHealth intervention is developed, we should take advantage of the work that has already been conducted, learn from it, and propel ourselves further forward in the process of creating the most useful, effective, and cost-efficient programs possible.

Funding

National Institutes of Health (T32HD068223 to Y.P.W.; K24HD060068 to T.M.P.; 5R01AR061513 to M.C.).

Conflict of interest: Dr. Ritterband has equity ownership in BeHealth Solutions, LLC, a company developing and making available eHealth related products. The terms of this arrangement have been reviewed and approved by the University of Virginia in accordance with its conflict of interest policy.

References

  1. Ahern D K, Patrick K, Phalen J M, Neiley J D. An introduction to methodological challenges in the evaluation of eHealth research: Perspectives from the Health e-Technologies Initiative. Evaluation and Program Planning. 2006;29:386–389. doi: 10.1016/j.evalprogplan.2006.08.008. doi:10.1016/j.evalprogplan.2006.08.008. [DOI] [PubMed] [Google Scholar]
  2. Anderson J G. Social, ethical and legal barriers to e-health. International Journal of Medical Informatics. 2007;76:480–483. doi: 10.1016/j.ijmedinf.2006.09.016. doi:10.1016/j.ijmedinf.2006.09.016. [DOI] [PubMed] [Google Scholar]
  3. Atkinson N L, Gold R S. The promise and challenge of eHealth interventions. American Journal of Health Behavior. 2002;26:494–503. doi: 10.5993/ajhb.26.6.10. doi:10.5993/AJHB.26.6.10. [DOI] [PubMed] [Google Scholar]
  4. Barak A. Psychological applications on the internet: A discipline on the threshold of a new millennium. Applied and Preventive Psychology. 1999;8:231–245. doi:10.1016/S0962-1849(05)80038-1. [Google Scholar]
  5. Bennett S, Maton K, Kervin L. The ‘digital natives’ debate: A critical review of the evidence. British Journal of Educational Technology. 2008;39: 775–786. doi:10.1111/j.1467-8535.2007.00793.x. [Google Scholar]
  6. Campbell L, Vasquez M J, Behnke S H, Kinscherff R. APA ethics code commentary and case illustrations. Portland, OR: American Psychological Association; 2010. [Google Scholar]
  7. Chiu T, Eysenbach G. Stages of use: Consideration, initiation, utilization, and outcomes of an internet-mediated intervention. BMC Medical Informatics and Decision Making. 2010;10:73. doi: 10.1186/1472-6947-10-73. doi:10.1186/1472-6947-10-73. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Christensen H, Griffiths K M, Farrer L. Adherence in internet interventions for anxiety and depression: Systematic review. Journal of Medical Internet Research. 2009;11:e13. doi: 10.2196/jmir.1194. doi:10.2196/jmir.1194. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Christensen H, Mackinnon A. The law of attrition revisited. Journal of Medical Internet Research. 2006;8:e20. doi: 10.2196/jmir.8.3.e20. doi:10.2196/jmir.8.3.e20. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Cushing C C, Jensen C D, Steele R G. An evaluation of a personal electronic device to enhance self-monitoring adherence in a pediatric weight-management program using a multiple baseline design. Journal of Pediatric Psychology. 2011;36:301–307. doi: 10.1093/jpepsy/jsq074. doi:10.1093/jpepsy/jsq074. [DOI] [PubMed] [Google Scholar]
  11. Cushing C C, Steele R G. A meta-analytic review of eHealth interventions for pediatric health promoting and maintaining behaviors. Journal of Pediatric Psychology. 2010;35:937–949. doi: 10.1093/jpepsy/jsq023. doi:10.1093/jpepsy/jsq023. [DOI] [PubMed] [Google Scholar]
  12. Curry S J. eHealth research and healthcare delivery: Beyond intervention effectiveness. American Journal of Preventive Medicine. 2007;32:S127–S130. doi: 10.1016/j.amepre.2007.01.026. doi:10.1016/j.amepre.2007.01.026. [DOI] [PubMed] [Google Scholar]
  13. Danaher B G, Seeley J R. Methodological issues in research on web-based behavioral interventions. Annals of Behavioral Medicine. 2009;38:28–39. doi: 10.1007/s12160-009-9129-0. doi:10.1007/s12160-009-9129-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Donkin L, Christensen H, Naismith S L, Neal B, Hickie I B, Glozier N. A systematic review of the impact of adherence on the effectiveness of e-therapies. Journal of Medical Internet Research. 2011;13:e52. doi: 10.2196/jmir.1772. doi:10.2196/jmir.1772. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Eng T R. eHealth research and evaluation: Challenges and opportunities. Journal of Health Communication. 2002;7:267–272. doi: 10.1080/10810730290001747. doi:10.1080/10810730290001747. [DOI] [PubMed] [Google Scholar]
  16. Ess C, AoIR Ethics Working Committee 2002 Ethical decision-making and Internet research. Retrieved from http://www.aoir.org/reports/ethics.pdf. Retrieved April 8, 2013. [Google Scholar]
  17. Eysenbach G. The law of attrition. Journal of Medical Internet Research. 2005;7:e11. doi: 10.2196/jmir.7.1.e11. doi:10.2196/jmir.7.1.e11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Eysenbach G. CONSORT-EHEALTH: Improving and standardizing evaluation reports of Web-based and mobile health interventions. Journal of Medical Internet Research. 2011;213:e126. doi: 10.2196/jmir.1923. doi:10.2196/jmir.1923. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Fatehi F, Wootton R. Telemedicine, telehealth or e-health? A bibliometric analysis of the trends in the use of these terms. Journal of Telemedicine and Telecare. 2012;18:460–464. doi: 10.1258/jtt.2012.gth108. doi:10.1258/jtt.2012.GTH108. [DOI] [PubMed] [Google Scholar]
  20. Fisher C B. Decoding the ethics code: A practical guide for psychologists. 2nd ed. Thousand Oaks, CA: Sage; 2009. [Google Scholar]
  21. Fox F E, Morris M, Rumsey N. Doing synchronous online focus groups with young people. Qualitative Health Research. 2007;17:539–547. doi: 10.1177/1049732306298754. doi:10.1177/1049732306298754. [DOI] [PubMed] [Google Scholar]
  22. Gustafson D, Wise M, Bhattacharya A, Pulvermacher A, Shanovich K, Phillips B, Lehman E, Chinchilli V, Hawkins R, Kim J S. The effects of combining web-based eHealth with telephone nurse case management for pediatric asthma control: A randomized controlled trial. Journal of Medical Internet Research. 2012;14:e101. doi: 10.2196/jmir.1964. doi:10.2196/jmir.1964. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Henderson E M, Law E F, Palermo T M, Eccleston C. Case study: Ethical guidance for pediatric e-health research using examples from pain research with adolescents. Journal of Pediatric Psychology. 2012;37:1116–1126. doi: 10.1093/jpepsy/jss085. doi:10.1093/jpepsy/jss085. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Jackson L A, Zhao Y, Kolenic A, III, Fitzgerald H E, Harold R, Von Eye A. Race, gender, and information technology use: The new digital divide. Cyberpsychology and Behavior. 2008;11:437–442. doi: 10.1089/cpb.2007.0157. doi:10.1089/cpb.2007.0157. [DOI] [PubMed] [Google Scholar]
  25. Kodish E. Informed consent for pediatric research: Is it really possible? Journal of Pediatrics. 2003;142:89–90. doi: 10.1067/mpd.2003.64. doi:10.1067/mpd.2003.64. [DOI] [PubMed] [Google Scholar]
  26. Kraut R, Olson J, Banaji M, Bruckman A, Cohen J, Couper M. Psychological research online: Report of Board of Scientific Affairs' Advisory Group on the Conduct of Research on the Internet. American Psychologist. 2004;59:105–117. doi: 10.1037/0003-066X.59.2.105. doi:10.1037/0003-066X.59.2.105. [DOI] [PubMed] [Google Scholar]
  27. Manhal-Baugus M. E-therapy: Practical, ethical, and legal issues. Cyberpsychology and Behavior. 2001;4:551–563. doi: 10.1089/109493101753235142. doi:10.1089/109493101753235142. [DOI] [PubMed] [Google Scholar]
  28. Norman G J, Zabinski M F, Adams M A, Rosenberg D E, Yaroch A L, Atienza A. A review of eHealth interventions for physical activity and dietary behavior change. American Journal of Preventive Medicine. 2007;33:336–345. doi: 10.1016/j.amepre.2007.05.007. doi:10.1016/j.amepre.2007.05.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Oh H, Rizo C, Enkin M, Jadad A. What is eHealth (3): A systematic review of published definitions. Journal of Medical Internet Research. 2005;7:e1. doi: 10.2196/jmir.7.1.e1. doi:10.2196/jmir.7.1.e1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Pagliari C. Design and evaluation in eHealth: Challenges and implications for an interdisciplinary field. Journal of Medical Internet Research. 2007;9:e15. doi: 10.2196/jmir.9.2.e15. doi:10.2196/jmir.9.2.e15. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Palermo T M. Editorial: Section on innovations in technology in measurement, assessment, and intervention. Journal of Pediatric Psychology. 2008;33:35–38. doi: 10.1093/jpepsy/jsm091. doi:10.1093/jpepsy/jsm091. [DOI] [PubMed] [Google Scholar]
  32. Palermo T M, Wilson A C, Peters M, Lewandowski A, Somhegyi H. Randomized controlled trial of an internet-delivered family cognitive–behavioral therapy intervention for children and adolescents with chronic pain. Pain. 2009;146:205–213. doi: 10.1016/j.pain.2009.07.034. doi:10.1016/j.pain.2009.07.034. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Riley W T, Glasgow R E, Etheredge L, Abernethy A P. Rapid, responsive, relevant (R3) research: A call for a rapid learning health research enterprise. Clinical and Translational Medicine. 2013;2:1–6. doi: 10.1186/2001-1326-2-10. doi:10.1186/2001-1326-2-10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Riley W T, Rivera D E, Atienza A A, Nilsen W, Allison S M, Mermelstein R. Health behavior models in the age of mobile interventions: Are our theories up to the task? Translational Behavioral Medicine: Practice, Policy, Research. 2011;1:53–71. doi: 10.1007/s13142-011-0021-7. doi:10.1007/s13142-011-0021-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Ritterband L M, Gonder-Frederick L A, Cox D J, Clifton A D, West R W, Borowitz S M. Internet interventions: In review, in use, and into the future. Professional Psychology: Research and Practice. 2003;34:527. doi:10.1037/0735-7028.34.5.527. [Google Scholar]
  36. Ritterband L M, Tate D F. The science of Internet interventions. Annals of Behavioral Medicine. 2009;38:1–3. doi: 10.1007/s12160-009-9132-5. doi:10.1007/s12160-009-9132-5. [DOI] [PubMed] [Google Scholar]
  37. Ritterband L M, Thorndike F P, Cox D J, Kovatchev B P, Gonder-Frederick L A. A behavior change model for Internet interventions. Annals of Behavior Medicine. 2009;38:18–27. doi: 10.1007/s12160-009-9133-4. doi:10.1007/s12160-009-9133-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Ritterband L M, Thorndike F P, Lord H R, Borowitz S M, Walker L S, Ingersoll K S, Sutphen J, Cox D J. An RCT of an internet intervention for pediatric encopresis with one-year follow-up. Clinical Practice in Pediatric Psychology. 2013;1:68–80. doi: 10.1037/cpp0000007. doi:10.1037/cpp0000007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Schueller S M, Muñoz R F, Mohr D C. Realizing the potential of behavioral intervention technologies. Current Directions in Psychological Science. 2013;22:478–483. doi:10.1177/0963721413495872. [Google Scholar]
  40. Sellers T A, Caporaso N, Lapidus S, Petersen G M, Trent J. Opportunities and barriers in the age of team science: Strategies for success. Cancer Causes Control. 2006;17:229–237. doi: 10.1007/s10552-005-0546-5. doi:10.1007/s10552-005-0546-5. [DOI] [PubMed] [Google Scholar]
  41. Steele R G, Wu Y P, Cushing C C, Jensen C D. Evaluation of child health matters a web-based tutorial to enhance school nurses’ communications with families about weight-related health. The Journal of School Nursing. 2012;29:151–160. doi: 10.1177/1059840512446070. doi:10.1177/1059840512446070. [DOI] [PubMed] [Google Scholar]
  42. Stinson J, White M, Isaac L, Campbell F, Brown S, Ruskin D, Karim A. Understanding the 60 information and service needs of young adults with chronic pain: Perspectives of young adults and their providers. Clinical Journal of Pain. 2013;26:600–612. doi: 10.1097/AJP.0b013e31826dce65. doi:10.1097/AJP.0b013e31826dce65. [DOI] [PubMed] [Google Scholar]
  43. Stinson J, Wilson R, Gill N, Yamada J, Holt J. A systematic review of internet-based self-management interventions for youth with health conditions. Journal of Pediatric Psychology. 2009;34:495–510. doi: 10.1093/jpepsy/jsn115. doi:10.1093/jpepsy/jsn115. [DOI] [PubMed] [Google Scholar]
  44. Tate D F, Finkelstein E A, Khavjou O, Gustafson A. Cost effectiveness of internet interventions: Review and recommendations. Annals of Behavioral Medicine. 2009;38:40–45. doi: 10.1007/s12160-009-9131-6. doi:10.1007/s12160-009-9131-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. The British Psychological Society. 2007 Conducting Research on the Internet. Supplementary guidance document to the BPS Code of Ethics and Conduct, focusing on additional ethical and practice issues inherent in internet mediated research. Retrieved from http://www.bps.org.uk/the-society/code-of-conduct/code-of-conduct_home.cfm. Retrieved April 7, 2013. [Google Scholar]
  46. van Gemert-Pijnen J, Nijland N, van Limburg M, Ossebaard H C, Kelders S M, Eysenbach G, Seydel E R. A holistic framework to improve the uptake and impact of eHealth technologies. Journal of Medical Internet Research. 2011;13:e11. doi: 10.2196/jmir.1672. doi:10.2196/jmir.1672. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. van Limburg M, van Gemert-Pijnen J, Nijland N, Ossebaard H C, Hendrix R, Seydel E R. Why business modeling is crucial in the development of eHealth technologies. Journal of Medical Internet Research. 2011;13:e124. doi: 10.2196/jmir.1674. doi:10.2196/jmir.1674. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Webb T L, Joseph J, Yardley L, Michie S. Using the Internet to promote health behavior change: A systematic review and meta-analysis of the impact of theoretical basis, use of behavior change techniques, and mode of delivery on efficacy. Journal of Medical Internet Research. 2010;12:e4. doi: 10.2196/jmir.1376. doi:10.2196/jmir.1376. [DOI] [PMC free article] [PubMed] [Google Scholar]