The rising prevalence and changing age distribution of multiple sclerosis in Manitoba
Citation Manager Formats
Make Comment
See Comments
This article has a correction. Please see:

Abstract
Objective: Several studies suggest an increasing prevalence of multiple sclerosis (MS) in Canada. We aimed to validate a case definition for MS using administrative health insurance data, and to describe the incidence and prevalence of MS in Manitoba, Canada.
Methods: We used provincial administrative claims data to identify persons with demyelinating disease using International Classification of Diseases 9/10 codes and prescription claims. To validate the case definition, questionnaires were mailed to 2,000 randomly selected persons with an encounter for demyelinating disease, requesting permission for medical records review. We used diagnoses abstracted from medical records as the gold standard to evaluate candidate case definitions using administrative data.
Results: From 1984 to 1997, cases of MS using claims data were defined as persons with ≥7 medical contacts for MS. From 1998 onward, cases were defined as persons with ≥3 medical contacts. As compared to medical records, this definition had a positive predictive value of 80.5% and negative predictive value of 75.5%. From 1998 to 2006, the average age- and sex-adjusted annual incidence of MS per 100,000 population was 11.4 (95% confidence interval [CI] 10.7–12.0). The age-adjusted prevalence of MS per 100,000 population increased from 32.6 (95% CI 29.4–35.8) in 1984 to 226.7 (95% CI 218.1–235.3) in 2006, with the peak prevalence shifting to older age groups.
Conclusion: The prevalence of multiple sclerosis (MS) in Manitoba is among the highest in the world. The rising prevalence with minimally changing incidence suggests improving survival. This study supports the use of administrative data to develop case definitions and further define the epidemiology of MS.
Glossary
- CI=
- confidence interval;
- DIN=
- drug identification number;
- DPIN=
- Drug Programs Information Network;
- ICD=
- International Classification of Diseases;
- MHHL=
- Manitoba Health and Healthy Living;
- MS=
- multiple sclerosis.
Multiple sclerosis (MS) is a chronic demyelinating disease of the CNS, affecting more than 1 million persons worldwide.1,2 MS adversely affects employment, social relationships, and quality of life.3,4 In 2006, a US study estimated the total (direct and indirect) mean annual cost per patient at $47,215.
Several studies suggest an increasing prevalence of MS in eastern and western regions of Canada,5–7 a nation recognized to have a high prevalence of MS.8 The Canadian province of Manitoba is centrally located, yet the incidence and prevalence of MS have not been estimated there since 1960.9 The comprehensive Manitoba health insurance databases provide a unique opportunity to conduct current population-based studies of MS.
We aimed to validate a definition for ascertaining cases of MS using administrative health insurance data and to apply this definition to describe the incidence and prevalence of MS in Manitoba.
METHODS
Manitoba is a central Canadian province with a stable population of nearly 1.2 million, 98% of whom receive health care coverage through the provincial government department, Manitoba Health and Healthy Living (MHHL).10 Since 1984, all residents have a unique personal health identification number which is attached to all health services claims submitted to MHHL, which maintains computerized records of these claims.
Each physician claim includes the patient's identification number, date of service, and 3-digit International Classification of Diseases (ICD)-9-CM code for the physician-assigned diagnosis. Each hospitalization record includes the patient's identification number, dates of admission and discharge, and up to 16 diagnostic codes listed on the discharge abstract using ICD-10-CA codes. Prior to 2004, these diagnoses were recorded using 5-digit ICD-9-CM codes. The Drug Programs Information Network (DPIN) is a pharmaceutical database capturing all outpatient prescription drug dispensations including the date, drug name, and drug identification number (DIN) for all residents, regardless of final payor. MHHL also maintains a population registry, which is updated when an individual moves into or out of Manitoba, changes marital or family status, or dies.
To identify residents with possible MS, we searched all physician claims and hospital discharge abstracts from April 1, 1984, through March 31, 2007, for diagnostic codes (ICD-9-CM/ICD-10-CA) for demyelinating diseases of the CNS, including MS and important variants of MS. The diagnostic codes included optic neuritis (377.3/H46), acute transverse myelitis (323.82/G37), acute disseminated encephalomyelitis (323/G36.9), demyelinating disease of CNS unspecified (341.9/G37.8), other acute disseminated demyelination (G36), MS (340/G35), and neuromyelitis optica (341.0/G36.0). We also searched DPIN for MS-specific prescription drugs, including interferon-beta-1b (Betaseron), interferon-beta-1a (Avonex), interferon-beta-1a (Rebif), glatiramer acetate (Copaxone), and natalizumab (Tysabri).
Using these criteria for physician, hospital, and prescription claims, MHHL identified 16,189 individuals with possible MS. To generate a list of potential questionnaire respondents, this list was restricted to persons who were ≥18 years old as of January 1, 2007, alive and residing in Manitoba at the time of study initiation, and who had ≥3 separate physician claims, hospitalizations, or prescription claims between 1984 and 2007; or ≥1 claim for persons resident in 2004 or later; at least one of these claims had to be specifically for MS or neuromyelitis optica (n = 5,247). This effectively enriched the cohort for persons with MS, to ensure enough MS cases to validate our definition. We mailed a questionnaire to 2,000 persons randomly selected from this list, sending a second mailing to persons who did not respond to the first mailing.
Questionnaire and medical records review.
We developed a self-administered questionnaire which captured information regarding the primary diagnosis, investigations for MS, years of initial MS symptom onset and diagnosis, and comorbidities. We asked participants for consent to use their questionnaire data, to review their medical records for diagnostic validation, and to link this information to their administrative claims data.
Irrespective of self-reported diagnosis, we randomly selected 400 medical charts for a structured chart review from a list of consenting respondents. One of us (R.A.M.) abstracted charts assisted by a trained research assistant, blinded to questionnaire responses and claims data. Following this, one of us (R.A.M.) classified participants as definite MS, possible MS, or not MS, according to the 2005 International Criteria, or as neuromyelitis optica.11,12
Administrative case definition of MS.
We developed several administrative case definitions, varying the number of physician, hospital, and prescription claims required to classify a person as having MS. In 1998, a provincial program was developed for the care of persons with MS; physicians who provide care under this program do not submit fee-for-service, or “shadow billing” claims. Therefore, we developed one definition for years 1984–1997, and one definition for years 1998–2006. We compared the classification of study cases according to the administrative case definitions and to diagnoses based on self-report and medical records review by computing sensitivity, specificity, and positive and negative predictive value. For this purpose, possible MS cases were classified as “not MS.” We did not capture enough cases of neuromyelitis optica to validate a case definition for that condition.
Incidence and prevalence.
To estimate the incidence of MS, we examined longitudinal records of medical contacts for all individuals who met the administrative case definition for MS. We considered the date of the first medical contact for any of the diagnostic codes listed above to be the date of diagnosis. Because our case definition differed before and after 1998, we calculated incidence rates for the period 1998–2006 to ensure the use of a consistent case definition throughout. Using the year of diagnosis of incident cases, we calculated the annual crude and age- and sex-adjusted incidence rates using the mid-year population figures from the MHHL population registry for denominators in the calculations. Using the direct method,13 the results were age-adjusted to the 2001 Canadian population. We calculated 95% confidence intervals (CI) assuming a Poisson distribution. We examined temporal trends in the age-adjusted incidence and prevalence using a linear regression model.
Standard protocol approvals, registrations, and patient consents.
Ethics approval was obtained from the University of Manitoba Health Research Ethics Board. Approval for data access was obtained from the Manitoba Health Information Privacy Committee. Written informed consent was obtained from all participants.
RESULTS
After 2 mailings, 69 questionnaires were returned to MHHL because they were undeliverable (65), the individual was deceased (3), or the individual was resident in a personal care home (1). Several persons responded to MHHL (5) or the investigators (22) refusing participation, for a response rate of 668/1,931 (34.6%). The age and sex distribution of the sample is shown in table 1. Due to privacy regulations, we had limited ability to compare responders and nonresponders. Responders were more likely to be women (77.7% vs 72.0%, p < 0.007), but the mean (SD) age did not differ from that of nonresponders (50.6 [12.3] vs 49.4 [13.0] years, p = 0.14). Among responders, 594 (88.9%) agreed to medical records review and to linkage of their medical records to claims data.
Table 1 Age and sex distribution of the random sample selected for questionnaire distribution, excluding deceased persons and those with incorrect mailing addresses
We tested administrative case definitions of MS in our cohort of responders, all of whom had at least one medical contact for MS (table e-1 on the Neurology® Web site at www.neurology.org). Based on these findings and to balance competing needs for sensitivity and specificity of the definition, we classified individuals with a first medical contact for MS before 1997 as having MS if they had ≥7 separate medical contacts, including physician, hospital, or prescription claims for MS. Using medical records as the gold standard for a diagnosis of MS, this definition had a positive predictive value of 85.1% and negative predictive value of 74.4%. Individuals who had an initial medical contact after 1997 were classified as cases if they had ≥3 separate medical contacts for MS. This definition had a positive predictive value of 77.0% and negative predictive value of 76.1%. The positive predictive value of the combined definition was 80.5% while the negative predictive value was 75.5%. Increasing the required number of contacts increased the specificity of the case definition, while decreasing the number of contacts increased the sensitivity. Using self-reported diagnoses as the gold standard produced similar results, with 7 medical contacts beginning before 1997 having a sensitivity of 92.4%, specificity of 76.7%, positive predictive value of 91.6%, and negative predictive value of 78.6%, and 3 medical contacts after 1997 having a sensitivity of 90.2%, specificity of 55.9%, positive predictive value of 74.5%, and negative predictive value of 80.0%. Agreement between medical records and self-reported diagnoses of MS was substantial (kappa = 0.86; 95% CI 0.81–0.91).
Incidence.
Between 1998 and 2006, applying the administrative case definitions defined above to the entire administrative dataset, the average annual age-adjusted incidence rate per 100,000 population was 11.4 (95% CI 10.7–12.0). Among women, incidence peaked at age 35 to 39 years, followed by ages 40 to 44 years, then ages 30 to 34 years (table 2). Among men, incidence peaked at age 45–49 years, followed by ages 50–54 years, then ages 35–44 years. Women had a higher incidence of MS than men (RR 3.15, 95% CI 2.75–3.60). Between 1998 and 2006, the annual incidence rate was essentially stable, with a change in incidence of −0.38 per 100,000 population per year (95% CI −0.80 to 0.028) (figure 1).
Table 2 Average annual incidence (AAI) of multiple sclerosis in Manitoba per 100,000 population by age and sex, 1998–2006
Figure 1 Age-standardized incidence of multiple sclerosis in Manitoba per 100,000 population from 1998 to 2006
Prevalence.
In 2006, the crude prevalence of MS was 278.0 per 100,000 population. The age-adjusted prevalence was 226.7 (95% CI 218.1–235.3), and was higher in women than men (RR 2.96; 95% CI 2.71–3.24). Among women, the age-adjusted prevalence was 340.0 (95% CI 325.3–354.6). Among men, the age-adjusted prevalence was 112.2 (95% CI 103.4–121.0). Among women, the peak prevalence was observed among persons aged 45 to 59 years, about 10 to 20 years later than the peak incidence. Among men, the peak prevalence was observed among persons aged 55–69 years (table e-2). With advancing age beyond these age groups, the prevalence declined steadily.
Regardless of the case definition used, the prevalence of MS in Manitoba increased steadily from 1984 onward. Using the definition requiring ≥7 hospital, physician, or prescription claims for MS, the prevalence of MS increased 8.14 (95% CI 7.59–8.69) per 100,000 population per year. Using the definition that required ≥3 hospital, physician, or prescription claims for MS, the prevalence of MS increased 9.4 (95% CI 8.61–10.2) per 100,000 population per year from 1998 to 2006, exceeding the increase in annual incidence (figure 2).
Figure 2 Age-standardized prevalence of multiple sclerosis in Manitoba per 100,000 population from 1984 to 2006
In later years, the peak age-specific prevalence shifted to older age groups (figure 3). Regardless of the case definition used, in 1984 the peak prevalence was observed among persons aged 35 to 39 years, and no person with MS over age 64 years was identified. Twenty years later, the peak prevalence was observed among persons aged 50 to 54 years. Persons over age 60 years were increasingly represented in later years, and MS cases were identified among persons aged 65 to 85 years (table e-2).
Figure 3 Age-specific prevalence of multiple sclerosis in Manitoba per 100,000 population by year between 1984 and 2004
When we compared the year of diagnosis based on medical records to the year of diagnosis from administrative data, they agreed within 3 years in 74.2% of cases. The year of diagnosis from administrative data agreed with the patient-reported year of diagnosis within 3 years in 76.3% of cases. Based on medical records, we examined the diagnostic delay between the year of first MS symptom onset and the year of diagnosis. The diagnostic delay decreased with a later year of symptom onset (r = −0.47, p < 0.0001). Based on medical records review, the mean (SD) diagnostic delay for persons with onset of MS symptoms between 1969 and 1978 was 12.3 (12.3) years, while the mean delay between 1979 and 1988 was 8.5 (7.6) years, between 1989 and 1998 was 4.0 (4.2) years, and between 2000 and 2008 was 1.5 (1.8) years.
DISCUSSION
Based on patient records and death certificates for 1939 to 1948, the prevalence of MS in Winnipeg, Manitoba, was 39.6 per 100,000.14 In 1960, a follow-up study in Winnipeg reported a stable prevalence of 35.4 per 100,000.9 We found a steady increase in MS prevalence in Manitoba from 1998 through 2006; the prevalence in 2006 is several-fold higher than previously reported. Given the increasing use of costly therapies for MS,15,16 the potential increased burden on health care services is substantial.
Several studies suggest an increasing prevalence of MS worldwide, including in Canada.5,17,18 In Newfoundland, the prevalence increased from 55 per 100,000 in 1960–1984 to 94 in 1996–2003.5,6 In Saskatoon, Saskatchewan, the prevalence was 111 per 100,000 in 1997 and 298 in 2005.17,19 In Alberta, a study using administrative claims data defined persons with MS as a person with 2 or more physician claims for MS or 1 or more hospital claims for MS; the prevalence of disease increased from 263 in 1994 to 322 in 2000.7 In 2001, a study in Nova Scotia using a similar case definition found a prevalence of 218 per 100,000. Applying a case definition requiring ≥2 hospital or physician claims in Manitoba from 1984 onward, the prevalence of MS would be 189 in 1994 and 272 in 2000. While we cannot exclude differences in health care utilization and billing practices as causes of the differences in these prevalence estimates, this is consistent with regional variation in disease burden across Canada.8
Prevalence is a function of incidence and disease duration. Overall, we observed a stable MS incidence rate, consistent with reports in other Canadian regions.5,6,17,19 Thus the rising prevalence of MS in Manitoba implies a longer disease duration, due to earlier age at symptom onset, earlier diagnosis, or later age at death. Another possibility is that MS incidence increased in the recent past then stabilized, so that a group of patients is still moving through the age groups and prevalence has not stabilized yet. Our data suggest that both earlier diagnosis and later age at death may be relevant. The diagnostic delay between the initial onset of MS symptoms and diagnosis decreased with later year of symptom onset, as observed elsewhere.20 The shift in peak prevalence to older ages and the rise in the prevalence of MS among persons in older age groups suggest that patients with MS are living longer. Studies in Denmark and Australia similarly suggest that the age at which death occurs in MS is increasing.21,22 This has important implications for patients and health care providers. As patients with MS age they are at increased risk for developing comorbidities,23 at a time when they are also likely to be experiencing increased disability. This is likely to increase the amount and complexity of their health care needs.24
In Canada and other countries with national payor systems, administrative health care databases are population-based, accessible, and cost-effective for research. Because these data are not collected for research purposes, however, their validity for research must be assessed.25 Administrative claims data may underrepresent chronic diseases as compared to medical records,26,27 but claims data outperform review of the records of a single provider,28 circumventing the problems of chart review when patients may have multiple providers. Once their validity are established for a disease, administrative data may provide a less costly method of disease surveillance than traditional studies which rely on multiple sources of ascertainment and medical records review; this approach is being used nationally in Canada to track diabetes and other diseases.29 Other uses for these databases include studies of etiology, health outcomes, and health care utilization.
We validated a case definition for MS using administrative claims data. Among the subset of persons with at least one health care claim for MS, this definition had a positive predictive value of 80.5% and a negative predictive value of 75.5%. If we assume that persons who never have a health care claim for any diagnostic code suggesting demyelinating disease have not been diagnosed with MS, then the specificity of our case definition in Manitoba's general population is greater than 99%. For studies where it is critical to include persons highly likely to have MS, the definition used should maximize specificity by increasing the number of health encounters required to be classified as MS. For studies where burden of disease and health care utilization are of greatest interest, a more sensitive case definition which requires fewer health care encounters may be more appropriate.
Other investigators used Veterans Health Administration claims data to develop a case definition for MS, with a sensitivity of 93% and specificity of 92%, similar at the population level to our case definition before 1998,30 but with a higher sensitivity as compared to our definition after 1998. The decrease in the sensitivity of administrative claims data for identifying cases of MS after 1998 when physicians participating in the provincial program for MS stopped providing care on a fee-for-service basis and the differences in case definition performance from one health care system to another illustrate the potential limitations of such data, and the need to formally validate administrative case definitions for disease and the need to re-evaluate these changes when changes in billing practice or health care utilization occur. This change may have led us to underestimate the burden of MS in our province; however, the general temporal changes in disease burden were unaffected by the case definition used.
Other limitations of this study should be noted. Our response rate was less than desired, but consistent with other studies in Manitoba using a random population-based mailing in which direct contact between potential participants and the investigators was not permitted.31 Responders were more likely to be women, but we do not know whether their health care utilization differed; if nonresponders differed from responders with respect to their frequency of utilization then our case definitions may not have the same predictive values in those individuals, or in men. Patients who are not followed by physicians or with infrequent medical contacts will be missed using our method, but an administrative database spanning more than 20 years makes this unlikely to be a significant source of bias. Further, the mean time between health care contacts for MS was less than 1 year.
Our study confirms that Manitoba is a region with a high and rising burden of MS, emphasizing the need to identify etiologic factors. Administrative claims data can be used for surveillance of chronic diseases, and our approach could be extended across provinces to develop national estimates of disease burden.
AUTHOR CONTRIBUTIONS
Statistical analysis was conducted by Stella Leung and Dr. Ruth Ann Marrie.
DISCLOSURE
Dr. Marrie serves on the editorial board of Neurology®; and receives research support from BioMS Medical, Sanofi-Aventis, Bayer Schering Pharma (Berlex), EMD Serono Canada, the Manitoba Health Research Council, Multiple Sclerosis Society of Canada, the Multiple Sclerosis Scientific Foundation, and the Consortium of Multiple Sclerosis Centers. Dr. Yu receives research support from the Canadian International Development Agency, the Multiple Sclerosis Society of Canada, and Manitoba Health and Healthy Living. Dr. Blanchard receives research support from the Canadian Institutes of Health Research, the Multiple Sclerosis Society of Canada, the Bill & Melinda Gates Foundation, Canadian International Development Agency, the World Bank, and the United States Agency for International Development. S. Leung reports no disclosures. Dr. Elliott has served on a scientific advisory board for Gilead Sciences, Inc. and receives research support from the Manitoba Health Research Council, the Canadian Institutes of Health Research, Health Sciences Centre Foundation, and the Multiple Sclerosis Society of Canada.
Footnotes
-
Supplemental data at www.neurology.org
e-Pub ahead of print on January 13, 2010, at www.neurology.org.
Study funding: Supported by the Research Grants Program (University of Manitoba), the Health Sciences Centre Department of Research and Foundation, and a Rudy Falk Clinician Scientist Award (to R.A.M.).
Disclosure: Author disclosures are provided at the end of the article.
Received July 4, 2009. Accepted in final form October 28, 2009.
REFERENCES
- ↵
Dean G. How many people in the world have multiple sclerosis? Neuroepidemiology 1994;13:1–7.
- ↵
Nortvedt MW, Riise T, Myhr K-M, Nyland HI. Quality of life in multiple sclerosis: Measuring the disease effects more broadly. Neurology 1999;53:1098–1103.
-
Rao SM, Leo GJ, Ellington L, Nauertz T, Bernardin L, Unverzagt F. Cognitive dysfunction in multiple sclerosis II: Impact on employment and social functioning. Neurology 1991;41:692–696.
- ↵
- ↵
- ↵
Beck CA, Metz LM, Svenson LW, Patten SB. Regional variation of multiple sclerosis prevalence in Canada. Mult Scler 2005;11:516–519.
- ↵
- ↵
Health Information Management Branch. Population report. Winnipeg, Manitoba: Manitoba Health and Healthy Living; 2008.
- ↵
-
Wingerchuk DM, Lennon VA, Pittock SJ, Lucchinetti CF, Weinshenker BG. Revised diagnostic criteria for neuromyelitis optica. Neurology 2006;66:1485–1489.
- ↵
Rothman KJ, Greenland S, eds. Modern Epidemiology, 2nd ed. Philadelphia, PA: Lippincott Williams & Wilkins; 1998.
- ↵
- ↵
Kobelt G, Berg J, Atherly D, Hadjimichael O. Costs and quality of life in multiple sclerosis: a cross-sectional study in the United States. Neurology 2006;66:1696–1702.
-
Federation MSI. Atlas: Multiple Sclerosis Resources in the World 2008. Geneva, Switzerland: World Health Organization; 2008.
- ↵
Hader WJ, Yee IML. Incidence and prevalence of multiple sclerosis in Saskatoon, Saskatchewan. Neurology 2007;69:1224–1229.
-
Grytten N, Glad SB, Aarseth JH, Nyland H, Midgard R, Myhr K-M. A 50-year follow-up of the incidence of multiple sclerosis in Hordaland County, Norway. Neurology 2006;66:182–186.
- ↵
Marrie RA, Cutter G, Tyry T, Hadjimichael O, Campagnolo D, Vollmer T. Changes in the ascertainment of multiple sclerosis. Neurology 2005;65:1066–1070.
- ↵
-
Bronnum-Hansen H, Koch-Henriksen N, Stenager E. Trends in survival and cause of death in Danish patients with multiple sclerosis. Brain 2004;127:844–850.
- ↵
Marrie RA, Horwitz R, Cutter G, Tyry T, Campagnolo D, Vollmer T. Comorbidity, socioeconomic status, and multiple sclerosis. Mult Scler 2008;14:1091–1098.
- ↵
Broemeling A, Watson DE, Prebtani F, on behalf of the Councillors of the Health Outcomes Steering Committee of the Health Council of Canada. Population patterns of chronic health conditions, co-morbidity and health care use in Canada: implications for policy and practice. Healthcare Q 2008;11:70–76.
- ↵
- ↵
- ↵
Hux JE, Ivis F, Flintoft V, Bica A. Diabetes in Ontario: determination of prevalence and incidence using a validated administrative data algorithm. Diabetes Care 2002;25:512–516.
- ↵
National Diabetes Surveillance System. Responding to the Challenge of Diabetes in Canada. Ottawa, Ontario: Health Canada; 2003.
- ↵
- ↵
Letters: Rapid online correspondence
REQUIREMENTS
You must ensure that your Disclosures have been updated within the previous six months. Please go to our Submission Site to add or update your Disclosure information.
Your co-authors must send a completed Publishing Agreement Form to Neurology Staff (not necessary for the lead/corresponding author as the form below will suffice) before you upload your comment.
If you are responding to a comment that was written about an article you originally authored:
You (and co-authors) do not need to fill out forms or check disclosures as author forms are still valid
and apply to letter.
Submission specifications:
- Submissions must be < 200 words with < 5 references. Reference 1 must be the article on which you are commenting.
- Submissions should not have more than 5 authors. (Exception: original author replies can include all original authors of the article)
- Submit only on articles published within 6 months of issue date.
- Do not be redundant. Read any comments already posted on the article prior to submission.
- Submitted comments are subject to editing and editor review prior to posting.
You May Also be Interested in
Hastening the Diagnosis of Amyotrophic Lateral Sclerosis
Dr. Brian Callaghan and Dr. Kellen Quigg
► Watch
Related Articles
Topics Discussed
Alert Me
Recommended articles
-
Article
Incidence and prevalence of MS in childrenA population-based study in Ontario, CanadaRuth Ann Marrie, Julia O'Mahony, Colleen Maxwell et al.Neurology, September 26, 2018 -
Review
Lower prevalence of multiple sclerosis in First Nations CanadiansRuth Ann Marrie, Stella Leung, Nancy Yu et al.Neurology: Clinical Practice, January 23, 2018 -
Article
Sex differences in comorbidity at diagnosis of multiple sclerosisA population-based studyRuth Ann Marrie, Scott B. Patten, Helen Tremlett et al.Neurology, March 09, 2016 -
Views & Reviews
A new way to estimate neurologic disease prevalence in the United StatesIllustrated with MSLorene M. Nelson, Mitchell T. Wallin, Ruth Ann Marrie et al.Neurology, February 15, 2019