A Brief Survey on Six Basic and Reduced eHealth Indicators in Seven Countries in 2017

Abstract

Background Holistic, ubiquitous support of patient-centered health care (eHealth) at all health care institutions and in patients' homes through information processing is increasingly supplementing institution-centered care. While eHealth indicators may measure the transition from institution-centered (e.g., hospital-centered) information processing to patient-centered information processing, collecting relevant and timely data for such indicators has been difficult.

Objectives This article aims to design some basic eHealth indicators, which are easily collected and measure how well information processing supports holistic patient-centered health care, and to evaluate penetrance of patient-centered health as measured by the indicators internationally via an expert survey.

Methods We identified six basic indicators that measure access of health care professionals, patients, and caregivers to the patient's health record data and the ability of providers, patients, and caregivers to add information in the patient's record. In a survey of international informatics experts, these indicators' penetrance were evaluated for Austria, Finland, Germany, Hong Kong, South Korea, Sweden, and the United States in the summer of 2017.

Results The eHealth status measured by the indicators varied significantly between these seven countries. In Finland, most practices measured by the indicators were fully implemented whereas in Germany only one practice was partially realized.

Conclusion Progress in the implementation of practices that support patient-centered care could mainly be observed in those countries where the “political will” focused on achieving patient-centered care as opposed to an emphasis on institution-centered care. The six eHealth indicators seem to be useful for measuring national progress in patient-centered care. Future work will extend the number of countries analyzed.

Keywords

health information exchanges - disease management (general) - health information systems - patient-centered care - connected health - health care - quality of care - international comparison

Background and Significance

Background

Health information systems have become more advanced during the last decades. Many of these advances were envisioned in the 1980s[1] for hospital information systems. Later, this concept was expanded to include visions for health information systems in the first decade of the 21st century[2] [3] and received further refinement by envisioning learning, self-correcting systems,[4] as well as expanding the model to include social determinants of health and patients' choices.[5]

The trend from institution-centered (e.g., hospital-centered) information processing for health care to information processing that supports patient-centered care across health care institutions was already discussed by Haux,[2] section 3.2 (see also fig. 6 on p. 277, line 2). Haux also discussed the need for patient-centered information technology for all health care professionals (section 3.3, see also fig. 6 on p. 277, line 3) as well as for patients. With the age of digitization bringing enormous computing power into consumer hands and growing interest in quantified health[6] [7] [8] and eHealth,[9] [10] [11] the demand and the need for information processing that supports patient-centered health care holistically and ubiquitously across health care institutions and patients' homes and work [12] continues to grow.

To measure progress on eHealth, the World Health Organization launched a Global Observatory for eHealth.[13] [14] [15] Several groups developed, refined, and discussed eHealth indicators to measure progress in eHealth including the Nordic eHealth Research Network.[16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] However, collecting relevant and timely data on a country's eHealth status through the measurement of well-elaborated eHealth indicators has been challenging especially when national authorities have to be contacted and queried.

Objectives

With the increasing need to determine the progress of information processing for patient-centered care and the challenges remaining in their implementation, the three objectives (O₁, O₂, O₃) of this work include:

• O₁ to develop basic eHealth indicators, which can easily be collected and measured (“Six Basic eHealth Indicators” section).

• O₂ to design a brief survey using these indicators to measure a country's eHealth status (“Survey Design and Organization” section).

• O₃ to run the survey using international informatics experts and to present its outcomes (“Survey Results” section).

Our report closes with a critical discussion of our approach, including its limitations (“Discussion” section).

For O₁, we could build on the intensive work on eHealth indicators, mentioned in the “Background” section, as well as on intensive discussions among the authors and colleagues, mentioned in the “Acknowledgments” section. Easy collection and measurement should directly lead to good validity and reliability as well as to good reproducibility.

Six Basic eHealth Indicators

The Six Basic eHealth Indicators

The basic indicators should have the following properties. They should

• be easy to collect and to measure,

• indicate the status and change over time of eHealth progress,

• support timely surveys requiring minimal workload,

• be patient outcome-oriented and therefore measuring patient benefits, and

• be reproducible and compliant with good scientific practice.

As targeted basic eHealth indicators, we identified six indicators (the index t stands for targeted), which are described in [Table 1].

Table 1 Six basic eHealth indicators

No.	Abbreviations	Indicator
1	AHt	Access of health care professionals to their patients' health record data
2	APt	Access of patients to their health record data
3	ACt	Access of caregivers to the patients' health record data
4	EHt	Enabling health care professionals to add data to their patients' health record(s)
5	EPt	Enabling patients to add data to their health record(s)
6	ECt	Enabling caregivers to add data to the patients' health record(s)

The term caregiver refers to those persons, informally taking care of patients. Mostly caregivers are parents, children, other relatives, or persons with close and long-term relationships to a patient. The term health record is used here in a very broad sense, as a record with data and documents generated as a byproduct of patient care.

Indicator 4, EH_t, in today's health information systems architectural styles often means: Data are added to the respective health record(s) of a health care professional's health care institution. Being able to do this is the base for obtaining positive indicators 1 to 3.

… Further Explained …

We defined the term “health record data” in AH, AP, and AC to include data from all health care institutions, where a patient received care. The modified indicators referring to the above terms (index e stands for further explained) are described [Table 2].

Table 2 Indicators for the terms further explained

No.	Abbreviations	Indicator
1	AHe	Access of health care professionals to their patients' health record data from all health care institutions, where the patient received care
2	APe	Access of patients to their health record data from all health care institutions, where the patient received care
3	ACe	Access of caregivers to the patients' health record data from all health care institutions, where the patient received care
4	EHe	Enabling health care professionals to add data to their patients' health record(s)
5	EPe	Enabling patients to add data to their health record(s)
6	ECe	Enabling caregivers to add data to the patients' health record(s)

… and Further Reduced

Even those basic eHealth indicators were still difficult to obtain for several reasons. Therefore, we further reduced the indicators' scope to make surveys feasible, allowing for easily collected outcome data and measures using the reduced indicators, while still satisfying the originally required properties.

For the three access indicators AH, AP, and AC, we limited

• “data” to “major relevant data” and defined these as data on diagnoses, on medication, and on problems (e.g., allergies).

For all six indicators, we limited

• “access” to “immediate access” and “add data” to “add data immediately,” with the assumption that immediate access can only occur through use of electronic health records (EHRs) and not through other means such as paper-based health records.

• “health care professionals” to three “selected groups of health care professionals” only including physicians, nurses, and pharmacists.

• “all health care institutions” to “selected health care institutions in (and only in) a respective country.” We defined major health care institutions as institutions for in- or outpatient treatment and care, such as hospitals, medical offices, nursing homes, outpatient nursing organizations, and pharmacies.

The final version of the reduced basic eHealth indicators (index r stands for further reduced) are described in [Table 3].

Table 3 Indicators for the terms explained and further reduced

No.	Abbreviations	Indicator
1	AHr	Immediate access of physicians, nurses, and pharmacists to their patients' electronic health records' major relevant data from selected health care institutions, where the patient received care, in a respective country
2	APr	Immediate access of patients to their electronic health records' major relevant data from selected health care institutions, where the patient received care, in a respective country
3	ACr	Immediate access of caregivers to the patients' electronic health records' major relevant data from selected health care institutions, where the patient received care, in a respective country
4	EHr	Enabling physicians, nurses, and pharmacists of selected health care institutions to add data immediately to their patients' electronic health record(s) in a respective country
5	EPr	Enabling patients to add data immediately to their electronic health record(s) in a respective country
6	ECr	Enabling caregivers to add data immediately to the patients' electronic health record(s) in a respective country

Survey Design and Organization

Target Date, Scope, Assessment, and Selected Countries

The survey collected data on information processing for patient-centered care to determine the eHealth status in seven countries on August 1, 2017.

Survey respondents (see the Authors' Contributions section) were instructed that their responses to the indicators could only include actual conditions (not plans or aspirations) for physicians, nurses, and pharmacists, or patients, or caregivers in the respective country. An indicator was considered not met when the functionality was clearly restricted (e.g., only available to patients with a certain disease or only available in a part of the respective country), or if the functionality had not been implemented or was in the planning stage.

For this initial survey, we selected a convenience sample of Austria, Finland, Germany, Hong Kong (as a special administrative region of China), South Korea, Sweden, and the United States. Information in these countries' health care systems and, in particular, on eHealth strategies of these countries can be found in World Health Organization “Atlas of eHealth country profiles”[15] and through links given there.

Questions, Outcome Values

Using the six basic eHealth indicators, we designed the following questions for the survey (index r stands for further reduced), which are described in [Table 4].

Table 4 Questions for the indicators

No.	Abbreviations	Question for the respective indicator
1	AHr	Can physicians, nurses, and pharmacists access their patients' electronic health records' major relevant data immediately from selected health care institutions in a respective country?
2	APr	Can patients access their electronic health records' major relevant data immediately from selected health care institutions in a respective country?
3	ACr	Can caregivers access the patients' electronic health records' major relevant data immediately from selected health care institutions in a respective country?
4	EHr	Can physicians, nurses, and pharmacists immediately add data to their patients' electronic health record(s) in a respective country?
5	EPr	Can patients immediately add data to their electronic health record(s) in a respective country?
6	ECr	Can caregivers immediately add data to the patients' electronic health record(s) in a respective country?

Please note that we did not map the phrase “where the patient received care” of the reduced basic indicators to the questions, as this meaning was already sufficiently clear.

To represent the outcomes for each of the indicators, we defined four values: given (++), partially given (+), not given (–), and not applicable (N/A).

In the event that three or more of the six indicators for a country were found to be given, we posed the following additional question to be considered: What are the major organizational (e.g., governance), legal, financial, and/or information system technology (e.g., architectural framework or standards) factors that resulted in achieving this advanced eHealth status in this country?

Conducting the Survey

Each author was responsible to determine the indicator values for her/his respective country. In September 2017, authors completed a questionnaire that included the questions delineated in the “Questions, Outcome Values” section. Subsequently, discussions between R.H. and each of the other authors clarified the meaning of responses and aided in the standard setting for the indicators' values. Authors revised and aligned their responses and values based on the discussions. In case of a ++ (fully given) or a – (not given) as outcome values, we decided that there was no necessity to further explain these results, as they can easily be reproduced and checked. In case of a + (partially given), we however saw the need to explain the rationale for this result. These explanations are documented in sections Access to Patient Data for Health Care Professionals, Access to Data for Patients and Their Caregivers, Enabling Health Care Professionals to Add Data, and Enabling Patients and Their Caregivers to Add Data, after presenting the respective indicator value tables. All authors reviewed the final revised questionnaires and all authors explicitly released the final version of their respective survey results. Complying with rules for good scientific practice (e.g., German Research Association[28]), all finally revised questionnaires were stored and may be requested from the corresponding author.

Indicator Calculation

Based on the outcome values discussed in the “Questions, Outcome Values” section, the six basic eHealth indicators were calculated (index r stands for further reduced) and are described in [Table 5].

Table 5 Indicator calculation

No.	Abbreviations	Indicator calculation
1	AHr	Calculation of this eHealth indicator is based on 3 × 7 = 21 outcome values: - Access to (1) diagnoses, (2) medication, and (3) problems - from hospitals (by (1) physicians, (2) nurses, (3) pharmacists), from (4) medical offices (by physicians), from (5) nursing homes (by nurses), from (6) outpatient nursing organizations (by nurses), and from (7) pharmacies (by pharmacists) Outcome values on access to diagnoses, medication, and problems from each health care institution by the selected health care professional were defined as follows: a. Access is ++ if all three outcome values for diagnoses, medication, and problems are ++ or if two are ++ and one is + b. Access is + if one outcome value is ++ and two are + or if all three outcome values are +; exception: for pharmacies a + was given if the outcome value for medication was + c. Access is – in all other cases Based on the then remaining 7 values, this eHealth indicator was calculated in the sequence: 1. Indicator is ++ if ≥ 4 outcome values are ++ 2. Indicator is + if ≥ 2 outcome values are ++ or ≥ 3 values are ++ or + 3. Indicator is – in all other cases
2, 3	APr, ACr	Calculation of these eHealth indicators is based on 3 × 5 = 15 outcome values: - Access to (1) diagnoses, (2) medication, and (3) problems - from (1) hospitals, (2) medical offices, (3) nursing homes, (4) outpatient nursing organizations, and (5) pharmacies For the values of diagnoses, medication, and problems for each selected health care institution, we used the same definition of values as for indicator AHr Based on the remaining 5 values, this eHealth indicator was calculated in the sequence: 1. Indicator is ++ if ≥ 3 outcome values are ++ and ≤ 2 outcome values are – 2. Indicator is + if ≥ 2 outcome values are ++ or + 3. Indicator is – in all other cases
4	EHr	This eHealth indicator is based on 7 outcome values: adding data in hospitals (by (1) physicians, (2) nurses, (3) pharmacists), in (4) medical offices (by physicians), in (5) nursing homes (by nurses), in (6) outpatient nursing organizations (by nurses), and in (7) pharmacies (by pharmacists)). It was calculated in the sequence: 1. Indicator is ++ if ≥ 4 outcome values are ++ 2. Indicator is + if ≥ 2 outcome values are ++ or ≥ 4 outcome values are at least + 3. Indicator is – in all other cases
5, 6	EPr, ECr	The outcome value itself served as eHealth indicator. No calculation was necessary

Survey Results

Result Presentation

In our survey, outcomes for major relevant data, that is, for diagnoses, medication, and problems (e.g., allergies), were very similar among similar institutions or for a specific group of health care professionals. Thus, we decided to present these results jointly allowing for the tables in this section to become easier to comprehend. Results for indicators 2 and 3 on accessing data as well as 5 and 6 on adding data were similar for patients and for caregivers. In countries, where this access was given to patients, caregivers used the same features after patients had given them permission. Thus, we present the results jointly for indicators 2 and 3 in the “Access to Data for Patients and Their Caregivers” section and for indicators 5 and 6 in the “Enabling Patients and Their Caregivers to Add Data” section.

Access to Patient Data for Health Care Professionals

We evaluated indicator 1 (AH_r) by using the question: “Can physicians, nurses, and pharmacists access their patients' EHRs' major relevant data immediately from selected health care institutions in a respective country?” The results are presented in [Table 6].

Table 6 Outcome for eHealth indicator 1 (AH) access of health care professionals to their patients' health record dat a as of August 1, 2017, for Austria (A), Finland (FIN), Germany (D), Hong Kong (HK), South Korea (ROK), Sweden (S), and the United States (USA)

		Country
Selected health care institutions	Selected groups of health care professionals	A	D	FIN	HK	ROK	S	USA
Hospitals	Physicians	+	–	++	++	–	++	+
	Nurses	+	–	++	++	–	++	+
	Pharmacists	–	–	++	++	–	+	+
Medical offices	Physicians	–	–	++	++	–	++	+
Nursing homes	Nurses	–	–	++	++	–	+	–
Outpatient nursing organization	Nurses	–	–	++	++	–	+	–
Pharmacies	Pharmacists	–	–	+	–	–	+	+
Indicator 1 AHr		+	–	++	++	–	+	+

Note: Please note that these are the outcomes for the reduced indicator, as defined in the “Six Basic eHealth Indicators” section.

Survey responders added the following additional explanations:

In Austria, information on diagnoses, medication, and problems is only available through discharge letters. In 2018, e-medication will be introduced throughout Austria.

In Finland, nursing homes and outpatient nursing organizations must have a medical leader. Physicians are therefore employed in these institutions and have access to health record data. Pharmacists in Finland have access through the Finnish National Data Repository KANTA.[29]

In Hong Kong, approximately 85% of the total hospital market is covered by the so-called Hospital Authority.[30] The Hospital Authority is a statutory body established in 1990 and is responsible for managing Hong Kong's public hospitals. It currently has a workforce of approximately 75,000 persons. The Hospital Authority currently manages 42 hospitals and institutions, 47 specialist outpatient clinics, and 73 general outpatient clinics.

In South Korea, visiting nurses from health care institutions and health centers perform outpatient nursing care.

In Sweden hospital, physicians and nurses can access health record data through the Swedish Summary Care Record (“Nationell Patientöversikt”). Provided by the Swedish eHealth Agency,[31] pharmacists (in hospitals as well as in pharmacies) access medication data via a national medication list, which contains all prescribed drugs for a patient. The list is aggregated from data of all pharmacies for the last 15 months. Access for physicians and pharmacists to these data are dependent on patient consent. Nurses can access data in emergency situations with patient consent. For nursing homes and outpatient nursing organizations, data access depends in Sweden on whether these community or privately based organizations are connected to the national health information exchange platform through which the data for the summary care record can be accessed. The status of connections is available at Anslutna vårdgivare.[32]

For the United States, access for hospitals, offices, and pharmacies was set to partially given, as in the United States interoperability exists in pockets. The Veterans Information Systems and Technology Architecture allows access to veterans' data from all over the country within its system. Regional health information exchanges like the Indiana Health Exchange[33] provide regional interoperability. Many medical offices are prepared to provide patient information via consolidated Clinical Document Architecture documents to tertiary centers when referring patients, but this communication depends on offices pushing the information as opposed to a data pull.[34] In addition, large electronic health care vendors are now permitting data exchange among their various clients resulting in an “in vendor product” interoperability. The vast majority of pharmacies in the United States use the vendor Surescripts for e-prescribing activities.[35]

Access to Data for Patients and Their Caregivers

Since the same tools were used to gain access for patients and caregivers (after receiving permission from patients), the results for indicators 2 and 3 were similar for both groups. Thus, we present indicators 2 and 3 jointly in this section.

The question to evaluate the reduced indicator 2 (AP_r) was: “Can patients access their EHRs' major relevant data immediately from selected health care institutions in a respective country?” The question for reduced indicator 3 (AC_r) was: “Can caregivers access the patients' EHRs' major relevant data immediately from selected health care institutions in a respective country?” The results for both indicators are presented in [Table 7].

Table 7 Outcome for eHealth indicators 2 (AP) access of patients to their health record data and 3 (AC) access of caregivers to the patients' health record dat a as of August 1, 2017, for Austria (A), Finland (FIN), Germany (D), Hong Kong (HK), South Korea (ROK), Sweden (S), and the United States (USA)

	Country
Selected health care institutions	A	D	FIN	HK	ROK	S	USA
Hospitals	+	–	++	–	++	+	+
Medical offices	–	–	++	–	++	+	+
Nursing homes	–	–	++	–	++	–	–
Outpatient nursing organizations	–	–	++	–	++	–	–
Pharmacies	–	–	++	–	+	+	–
Indicators 2 APr and 3 ACr	–	–	++	–	++	+	+

Note: Please note that these are the outcomes for the reduced indicator, as defined in the “Six Basic eHealth Indicators” section.

Survey responders added the following additional explanations:

In Austria, information from hospitals is only available through discharge letters.

In Finland, access is provided through the Finnish National Data Repository KANTA.[29]

In South Korea, access is available through the National Health Insurance Service's My Health Bank.[36] [37]

In Sweden, access for patients and caregivers to health record data in hospitals, offices, and pharmacies is provided through the National Patient Portal “1177” via the e-service “Journalen.” The indicators were set as partially given since the data are organized by county or region resulting in varying access and availabilities. Available data per county or region are listed in https://www.inera.se/globalassets/tjanster/journalen/inforande/bild-pa-anslutna-och-info-som-visas_2018-01-19.png.[38] In certain regions, patients were able to share electronic access to their health records with relatives but the National Court found this to be noncompliant with Swedish law in 2017; limiting caregiver access to parents of children under the age of 13.

In the United States, hospitals and medical offices have implemented EHRs that include patient portals. These portals allow patients and caregivers to see relevant medications, diagnosis lists, and allergies. Access of patients and caregivers was set to partially given, as patients have various providers resulting in a lack of aggregated data and the need for multiple portals.

Enabling Health Care Professionals to Add Data

The question for the reduced indicator 4 (EH_r) was: “Can physicians, nurses, and pharmacists immediately add data to their patients' EHR(s) in a respective country?” The results are presented in [Table 8].

Table 8 Outcome for eHealth indicator 4 (EH) enabling health care professionals to add data to their patients' health record(s) as of August 1, 2017, for Austria (A), Finland (FIN), Germany (D), Hong Kong (HK), South Korea (ROK), Sweden (S), and the United States (USA)

		Country
Selected health care institutions	Selected groups of health care professionals	A	D	FIN	HK	ROK	S	USA
Hospitals	Physicians	++	++	++	++	++	++	++
	Nurses	++	++	++	−	++	++	++
	Pharmacists	−	+	++	−	−	−	++
Medical offices	Physicians	++	++	++	++	++	++	++
Nursing homes	Nurses	−	−	++	−	++	++	+
Outpatient nursing organization	Nurses	−	−	++	−	++	++	+
Pharmacies	Pharmacists	−	+	+	−	−	−	++
Indicator 4 EHr		+	+	++	+	++	++	++

Note: Please note that these are the outcomes for the reduced indicator, as defined in the “Six Basic eHealth Indicators” section.

Survey responders added the following additional explanations:

In Finland, pharmacy-based pharmacists can add notifications to e-prescriptions.

In Germany, pharmacists in hospitals can indirectly add data through medication recommendations to physicians in addition to their own (but separate) documentation. For pharmacy-based pharmacists, the indicator is only partially given, because data are only stored for selected patients and at selected pharmacies.

In Hong Kong, nurses can enter nursing discharge summaries and subsequent care plans, although data are not recorded in real time with care delivery.

In the United States, not all nursing homes and all nursing outpatient organizations use EHRs.

Enabling Patients and Their Caregivers to Add Data

The results for indicators 5 and 6 were similar for patients and for caregivers in countries, where adding data was possible (Finland, Sweden, the United States). After receiving permission, caregivers would use the same access as patients. Thus, we present indicators 5 and 6 jointly in this section.

The question for reduced indicator 5 (EP_r) was: “Can patients immediately add data to their EHR(s) in a respective country?” The question for reduced indicator 6 (EC_r) was: “Can caregivers immediately add data to the patients' EHR(s) in a respective country?” The results for both indicators are presented in [Table 9].

Table 9 Outcome for eHealth indicators 5 (EP) enabling patients to add data to their health record(s) and 6 (EC) enabling caregivers to add data to the patients' health record(s) as of August 1, 2017, for Austria (A), Finland (FIN), Germany (D), Hong Kong (HK), South Korea (ROK), Sweden (S), and the United States (USA)

		Indicator	Country
No.	Abbreviations	Name	A	D	FIN	HK	ROK	S	USA
5 6	EPr ECr	Enabling patients to add data to their health record(s) Enabling caregivers to add data to the patients' health record(s)	–	–	+	–	–	+	+

Note: Please note that these are the outcomes for the reduced indicator, as defined in the “Six Basic eHealth Indicators” section.

Survey responders added the following additional explanations:

In Finland, patients' EHRs are comprised of multiple records and patients can add information to some of them. Further, in the national data repository, citizens have their own “My KANTA pages,”[39] where they can request online services. Attached to the Finnish National Data Repository KANTA[29] is a “Personal Health Record,”[40] which is a national data repository, where citizens may enter voluntary information on their health and well-being including citizens' measurements (e.g., weight), lifestyle, and activity. Caregivers may use this feature if the patient gave permission. Data, added by either patients or caregivers, are stored in the patients' files without attribution making a distinction of data from patients or caregivers hardly possible.

In Sweden, patients can add comments to their EHR and complete predefined questionnaires by providers but only in some regions. Adding data by caregivers (after patient approval) was possible in some regions until the end of 2017. This function is now only available for parents of children under the age of 13.

In the United States, most physicians and hospitals allow patients and caregivers to provide limited data and information to the EHR including scheduling and medication refill requests. Some portals allow patients to record blood pressure or blood glucose or document symptoms.

Summary Table

A summary of all eHealth indicator outcomes is listed in [Table 10].

Table 10 Summary table for the outcomes for the six basic and reduced eHealth indicators as of August 1, 2017, for Austria (A), Finland (FIN), Germany (D), Hong Kong (HK), South Korea (ROK), Sweden (S), and the United States (USA)

		eHealth indicator	Country
No.	Abbreviations	Name	A	D	FIN	HK	ROK	S	USA
1	AHr	Access of health care professionals to their patients' health record data	+	–	++	++	–	+	+
2	APr	Access of patients to their health record data	–	–	++	–	++	+	+
3	ACr	Access of caregivers to the patients' health record data	–	–	++	–	++	+	+
4	EHr	Enabling health care professionals to add data to their patients' health record(s)	+	+	++	+	++	++	++
5	EPr	Enabling patients to add data to their health record(s)	–	–	+	–	–	+	+
6	ECr	Enabling caregivers to add data to the patients' health record(s)	–	–	+	–	–	+	+

Note: Please note that these are the outcomes for the reduced indicator, as defined in the “Six Basic eHealth Indicators” section.

Discussion

The outcome values of the six eHealth indicators varied significantly among the seven included countries indicating significant eHealth disparities as defined by us. In Finland, all indicator values were fully given for most indicators and at least partially for the rest, while, for example, in Germany only one outcome value was partially given. While we were aware of differences in national values like the importance of data exchange and privacy, we did not anticipate the large variability. The eHealth indicators are measuring the quality of shared, integrated patient-centered care beyond one health care institution. These indicators do not measure the quality of care in a particular health care institution. Although Germany, for example, has rather poor outcomes in this survey, patient care in Germany is on a high level. For example, in the Global Burden of Disease Study 2015, Germany received a “Healthcare Access and Quality Index” of 86 (maximum index: 100, minimum: 0) and was (as Austria, Finland, South Korea, and Sweden) among the 25 best rated countries out of 195 countries.[41] However, a focus on institution-centered care and not patient-centered, shared care across institutions can disadvantage patient care through duplication of tests and services as well as limit research in biomedicine and in health sciences.

When we initiated the survey, we considered the indicator definitions as unambiguous and clear and anticipated that indicator values would be well reproducible. However, when conducting the survey, we realized that this was not always the case even with a reduced scope as defined in the “… and Further Reduced” section. Conducting the survey showed that different regulations and workflows resulted in different meanings and interpretations of terms in different countries. For example, the five types of health care institutions (hospitals, medical offices, nursing homes, outpatient nursing organizations, and pharmacies) included in this survey were not always comparable between countries. In Finland, nursing homes and outpatient nursing organizations require a physician supervisor. So, in addition to nurses there are also physicians employed in these institutions with access to patient data. In South Korea, most of the outpatient nursing care is done by visiting nurses from inpatient health care institutions and health centers rendering the term outpatient nursing “organization” unfit.

During the study, we realized that the results for indicators on accessing and adding data were similar for patients and for caregivers. In those countries, where this functionality was available, caregivers used the same access as the patients after receiving patients' permission.

Distinguishing between outcomes fully or partially available was not as easy as expected. It was necessary to discuss responses to align final values on the same scale and to achieve comparable results. We might have achieved a higher level of quality assurance if we had more than one respondent for each country to validate results. We plan to use multiple respondents for each country for future studies.

Nevertheless, users judged the indicators chosen as valuable by users and surveys using these indicators will fulfill the important criteria outlined in the “The Six Basic eHealth Indicators” section including ease of use, relevance, and reproducibility to comply with good scientific practice.

Also, eHealth is of course much broader than entering or accessing data, which is in the focus of the six indicators, presented here. Still, in our opinion, these indicators present a clear view on one important aspect on eHealth.

If at least three of the six indicators for a country were fulfilled, we followed up with the question: “What are the major organizational (e.g., governance), legal, financial, and/or information system technology (e.g., architectural framework or standards) factors that resulted in achieving this advanced eHealth status in this country?” At least three of the six indicators were fulfilled for Finland and South Korea. In our view, the main reason is that for both countries their clear political will achieved patient-centered care and not “just” institution-centered care. The unique social security number in Finland and the unique resident registration number in South Korea as a standard identifier acts as the key for health care databases. Privacy and security are monitored based on legislation. The organizational and information systems approaches in both countries were rather different and can hardly be regarded as major factors. We therefore plan future work analyzing the difference we found and will report on those results separately from this work on the eHealth indicators.

Three additional major limitations have to be mentioned: The austerity of the indicators, the dynamics of the eHealth developments, and the selection of countries. The desired indicator ease of use and reproducibility resulted in lack of coverage of other important aspects of care quality and efficiency. In addition, the six indicators did not consider usability of patient data for research. In many countries, national eHealth solutions are just being implemented. For example, in Austria, a national eHealth infrastructure is being introduced in 2018. Thus, many eHealth indicators will probably be fulfilled in one or two years. We thus plan to repeat this survey in regular time intervals. We intend to extend the number of countries in future surveys in a similar fashion: Measuring basic and reduced eHealth indicators using medical informaticians, who are independent, experienced in research and/or practice, and have been active internationally, within the International Medical Informatics Association (IMIA), related regional organizations, or within the member societies constituting IMIA.[42]

Clinical Relevance Statement

We reported on eHealth indicators for patient-centered care and we measured indicator values in seven countries. Our findings show that the eHealth status measured by the indicators varied significantly between these seven countries.

Multiple Choice Questions

1. Which eHealth indicator(s) is/are outcome-oriented and measuring patient benefits?

   • Degree of digitization of health care institutions in a certain country.

   • Existence of a national eHealth strategy.

   • Immediate access of health care professionals to their patients' diagnoses in a certain country.

   • All of the above.

   Correct Answer: The correct answer is option c, immediate access of health care professionals to their patients' diagnoses in a certain country. Such access is clearly supportive for patient care. Both options a and b are not measuring patient benefits and are therefore not outcome-oriented. Although they may serve as eHealth indicators.

2. Which of the following eHealth indicator(s) is/are easy to collect and to measure and so better reproducible and compliant with good scientific practice?

   • Relative number of patients who received wrong medication in a certain country.

   • Existence of a national eHealth strategy.

   • Patient satisfaction with timeliness and quality of care in a certain country.

   • All of the above.

   Correct Answer: The correct answer is option b, existence of a national eHealth strategy. Existence can easily be checked and is also easily reproducible. Although of importance, the outcome-oriented eHealth indicators mentioned in options a and c are, however, difficult to measure.

Conflict of Interest

None.

Acknowledgments

We want to thank Ursula Hübner from the Osnabrück University of Applied Sciences as well as Michael Marschollek and Klaus-Hendrik Wolf from the Peter L. Reichertz Institute for Medical Informatics for their advice and comments on our eHealth indicators.

Authors' Contributions

Each author is responsible for the values of indicators for her or his respective country. R.H. led the overall organization of this article and designed the survey. The six basic indicators were the outcome of discussions of R.H. with all authors, in particular with E.A. and S.K., who also contributed to the references.

Protection of Human and Animal Subjects

This article does not contain any studies with human participants or animals performed by any of the authors.

• References

• 1 Reichertz PL. Hospital information systems--past, present, future. Int J Med Inform 2006; 75 (3–4): 282-299
• 2 Haux R. Health information systems - past, present, future. Int J Med Inform 2006; 75 (3–4): 268-281
• 3 Kuhn KA, Giuse DA. From hospital information systems to health information systems. Problems, challenges, perspectives. Methods Inf Med 2001; 40 (04) 275-287
• 4 Friedman CP, Rubin JC, Sullivan KJ. Toward an information infrastructure for global health improvement. Yearb Med Inform 2017; 26 (01) 16-23
• 5 Gonzalez Bernaldo de Quiros F, Dawidowski AR, Figar S. Representation of people's decisions in health information systems. A complementary approach for understanding health care systems and population health. Methods Inf Med 2017; 56 (Open): e13-e19
• 6 Almalki M, Gray K, Martin-Sanchez FJ. Refining the concepts of self-quantification needed for health self-management. A thematic literature review. Methods Inf Med 2017; 56 (01) 46-54
• 7 Arnrich B, Ersoy C, Mayora O, Dey A, Berthouze N, Kunze K. Wearable therapy - detecting information from wearables and mobiles that are relevant to clinical and self-directed therapy. Methods Inf Med 2017; 56 (01) 37-39
• 8 Gray K, Martin-Sanchez FJ, Lopez-Campos GH, Almalki M, Merolli M. Person-generated data in self-quantification. A health informatics research program. Methods Inf Med 2017; 56 (01) 40-45
• 9 World Health Assembly. Fifty-eighth Assembly (May 16–25, 2005), eHealth Resolution WHA58.28. Available at: http://www.who.int/healthacademy/media/WHA58-28-en.pdf . Accessed March 3, 2018
• 10 Healy JC. The WHO eHealth Resolution - eHealth for all by 2015?. Methods Inf Med 2007; 46 (01) 2-4
• 11 Al-Shorbaji N. The World Health Assembly resolutions on eHealth: eHealth in support of universal health coverage. Methods Inf Med 2013; 52 (06) 463-466
• 12 Haux R. My home is my hospital. Stud Health Technol Inform 2016; 226: 3-8
• 13 World Health Organization. Global Observatory for eHealth. Available at: http://www.who.int/goe/en/ . Accessed August 31, 2017
• 14 Kay M, Santos J. Report on the World Health Organization Global Observatory for eHealth strategic planning workshop, April 2008. Methods Inf Med 2008; 47 (04) 381-387
• 15 World Health Organization. Atlas of eHealth country profiles. Based on the findings of the second global survey on eHealth. Geneva: WHO Press; 2011 . Available at: http://apps.who.int/iris/bitstream/10665/44502/1/9789241564168_eng.pdf . Accessed March 3, 2018
• 16 Gray BH, Bowden T, Johansen I, Koch S. Electronic health records: an international perspective on “meaningful use”. Issue Brief (Commonw Fund) 2011; 28: 1-18
• 17 Hyppönen H, Ammenwerth E, de Keizer N. Exploring a methodology for eHealth indicator development. Stud Health Technol Inform 2012; 180: 338-342
• 18 Hyppönen H, Ammenwerth E, Nohr C, Faxvaag A, Walldius A. eHealth indicators: results of an expert workshop. Stud Health Technol Inform 2012; 180: 328-332
• 19 Hyppönen H, Faxvaag A, Gilstad H. , et al. Nordic eHealth indicators: organisation of research, first results and plan for the future. Stud Health Technol Inform 2013; 192: 273-277
• 20 Hyppönen H, Faxvaag A, Gilstad H. , et al. Vimarlund. Nordic eHealth Indicators. Organisation of Research, first results and the plan for the future. Copenhagen: Nordic Council of Ministers; 2013 . Available at: http://norden.diva-portal.org/smash/get/diva2:700970/FULLTEXT01.pdf . Accessed March 3, 2018
• 21 Hyppönen H, Kangas M, Reponen J. , et al. Kaipio J. Nordic eHealth Benchmarking. Status 2014. Organisation of research, first results and the plan for the future. Copenhagen: Nordic Council of Ministers; 2015 . Available at: https://norden.diva-portal.org/smash/get/diva2:821230/FULLTEXT01.pdf . Accessed March 3, 2018
• 22 Hyppönen H, Koch S, Faxvaag A. , et al. Nordic eHealth benchmarking. From piloting towards established practice. Copenhagen: Nordic Council of Ministers; 2017 . Available at: https://norden.diva-portal.org/smash/get/diva2:1093162/FULLTEXT01.pdf . Accessed March 3, 2018
• 23 Hyppönen H, Ronchi E, Adler-Milstein J. Health care performance indicators for health information systems. Stud Health Technol Inform 2016; 222: 181-194
• 24 Moen A, Hackl WO, Hofdijk J. , et al. eHealth in Europe – status and challenges. Yearb Med Inform 2013; 8: 59-63
• 25 Stanimirović D, Vintar M. Development of eHealth at a national level - comparative aspects and mapping of general success factors. Inform Health Soc Care 2014; 39 (02) 140-160
• 26 Vedlūga T, Mikulskienė B. Stakeholder driven indicators for eHealth performance management. Eval Program Plann 2017; 63: 82-92
• 27 Zelmer J, Ronchi E, Hyppönen H. , et al. International health IT benchmarking: learning from cross-country comparisons. J Am Med Inform Assoc 2017; 24 (02) 371-379
• 28 German Research Association. Proposals for safeguarding good scientific practice. Bonn; 1998. Available at: http://www.dfg.de/download/pdf/foerderung/rechtliche_rahmenbedingungen/gute_wissenschaftliche_praxis/self_regulation_98.pdf . Accessed March 3, 2018
• 29 Finnish National Data Repository KANTA. Available at: http://www.kanta.fi/en/web/ammattilaisille/earkiston-esittely . Accessed March 3, 2018
• 30 Hospital Authority of Hong Kong. Available at: http://www.ha.org.hk/visitor/ha_index.asp . Accessed March 3, 2018
• 31 Swedish eHealth Agency. Available at: https://www.ehalsomyndigheten.se/other-languages/english/ . Accessed March 3, 2018
• 32 Anslutna vårdgivare [connected health services, only in Swedish]. Available at: https://www.inera.se/Fordjupning/fordjupad-information-om-vara-tjanster/Nationell-patientoversikt/anslutna-vardgivare-i-npo/ . Accessed March 3, 2018
• 33 Indiana Health Exchange. Available at: http://www.ihie.org/ . Accessed March 3, 2018
• 34 Lehmann CU, Kressly S, Hart WWC, Johnson KB, Frisse ME. Barriers to pediatric health information exchange. Pediatrics 2017; 139 (05) e20162653
• 35 Surescripts Network Alliance. Available at: http://surescripts.com/ . Accessed March 3, 2018
• 36 National Health Insurance Service. ‘My Health Bank’. Available at: https://sis.nhis.or.kr/jsp/o/f/ggof010m00_00.jsp . Accessed March 3, 2018
• 37 Lee J-W, Cho K-H. Overview of National PHR in Korea: Introduction of South Korea's National PHR Portal My Health Bank. Available at: http://www.ehealth.org.hk/APEHRC2016/Speaker-Abstract/D2%20Semi%201135%20Prof%20Kyung%20Hee%20Cho.pdf . Accessed March 3, 2018
• 38 Available at: https://www.inera.se/globalassets/tjanster/journalen/inforande/bild-pa-anslutna-och-info-som-visas_2018-01-19.png . Accessed March 3, 2018
• 39 My KANTA pages. Available at: http://www.kanta.fi/web/ammattilaisille/omakanta2 . Accessed March 3, 2018
• 40 Personal Health Record KANTA. . Available at: http://www.kanta.fi/web/ammattilaisille/omatietovaranto . Accessed March 3, 2018
• 41 GBD 2015 Healthcare Access and Quality Collaborators. Electronic address: [e-mail]; GBD 2015 Healthcare Access and Quality Collaborators. Healthcare Access and Quality Index based on mortality from causes amenable to personal health care in 195 countries and territories, 1990-2015: a novel analysis from the Global Burden of Disease Study 2015. Lancet 2017; 390 (10091): 231-266
• 42 International Medical Informatics Association (IMIA). Available at: http://imia-medinfo.org/wp/ . Accessed March 3, 2018