Int J Med Sci 2017; 14(10):920-926. doi:10.7150/ijms.19877 This issue Cite

Research Paper

Identification and Characteristics of Time-Related Shifts in Suicide-Related Event Frequency During Smoking Cessation Treatment with Varenicline

Hayato Akimoto1, Haruna Wakiyama1, Shinji Oshima1, Akio Negishi1, Kousuke Ohara1,2, Sachihiko Numajiri1, Mitsuyoshi Okita3, Shigeru Ohshima1, Naoko Inoue1, Daisuke Kobayashi1 Corresponding address

1. Faculty of Pharmaceutical Sciences, Josai University; 1-1 Keyakidai, Sakado, Saitama, 350-0295, Japan
2. Faculty of Pharmaceutical Sciences, Josai International University; 1 Gumyo, Togane, Chiba, 283-8555, Japan
3. Josai University Pharmacy, 909-4 Simogawara, Moroyama, Iruma, Saitama, 350-0435, Japan.

Citation:
Akimoto H, Wakiyama H, Oshima S, Negishi A, Ohara K, Numajiri S, Okita M, Ohshima S, Inoue N, Kobayashi D. Identification and Characteristics of Time-Related Shifts in Suicide-Related Event Frequency During Smoking Cessation Treatment with Varenicline. Int J Med Sci 2017; 14(10):920-926. doi:10.7150/ijms.19877. https://www.medsci.org/v14p0920.htm
Other styles

File import instruction

Abstract

Objectives: To survey time-related shifts in number of suicide-related events (SRE) during smoking cessation treatment with varenicline (VAR) in cases from the U.S. Food and Drug Administration (FDA) Adverse Event Reporting System (FAERS), as well as the characteristics of these shifts.

Methods: We isolated cases from the FAERS database involving VAR usage where SRE was reported as an adverse event (SRE+/VAR+ case) and established a histogram of SRE+/VAR+ case numbers per week. Furthermore, we focused on “cases reporting specific adverse events prior to drug usage start” using X-bar and R chart concepts. We also attempted to exclude the influence of smoking history from the created histogram. Moreover, we constructed a histogram on central nervous system adverse events, which were frequently seen during VAR usage.

Results: By removing the effects of smoking history, SRE onset signals were detected over a long period from the start of VAR use. However, expression signals for nausea and abnormal dreams were detected only in the early VAR administration period.

Discussion: These results suggest that VAR use-induced SRE is expressed over a long timeframe from the start of treatment. Additionally, the period of SRE expression signal detection was longer than that of the other central nervous system adverse events (nausea and abnormal dreams). Therefore, SRE onset must be carefully monitored during smoking cessation treatment with VAR over the entire treatment period.

Keywords: suicide-related event, varenicline, case study, FAERS, smoking cessation

Introduction

Smoking is a risk factor for cancer, chronic obstructive pulmonary disease (COPD), cardiovascular disease, and stroke. More than 6 million people die every year owing to the harmful effects of smoking on health [1]. In Japan alone, smoking is responsible for medical economic and socio-economic losses of approximately $15 billion and $20 billion, respectively [2], while in the U.S., medical economic and socio-economic losses are reported to be approximately $170 billion and $156 billion, respectively [3,4]. Smoking cessation treatment has a high cost-effectiveness in terms of preventing secondary diseases [5], and smoking cessation is recommended as part of a measure to reduce medical expenditure.

Currently, the α4β2 nicotinic receptor partial agonist varenicline (Chantix®, hereinafter referred to as VAR), and the noradrenaline and dopamine reuptake inhibitor bupropion (Zyban®) are clinically used as smoking cessation aids. The one-year abstinence rate for these drugs is 21.9% and 16.1%, respectively, which is higher than that for placebo [6]. Additionally, VAR use not only has a higher abstinence rate than that reported for nicotine replacement therapy and bupropion use, it is also less prone to nicotine withdrawal symptoms associated with smoking cessation [7, 8].

However, in 2009, the Food and Drug Administration (FDA) issued a Boxed Warning that VAR use may induce behavioural changes such as depression, suicidal ideation, and suicidal behaviour [9]. Case-control studies and case reports indicated that VAR may cause neuropsychiatric events [10-12]. In addition, in CzeekV, the Japanese search system for the FDA Adverse Event Reporting System (FAERS) (Accessed: 2016/10/05, version 3.0.1), VAR was ranked at number 2 among all drugs that induce suicidal ideation and suicidal behaviour (bupropion was ranked at number 35 for suicidal ideation and number 37 for suicidal behaviour) [13]. However, the results of randomised placebo-controlled trials and meta-analyses did not suggest that this drug directly caused these events [7, 14]. Thus, the risk of VAR causing neuropsychiatric events, including suicide-related events (SRE) such as suicidal ideation and suicidal behaviour, depends on study design. However, given that a Boxed Warning has been issued, it is essential to take precautions regarding the onset of neuropsychiatric events during VAR use.

We have already reported that VAR use increases SRE risk, which is the worst outcome among neuropsychiatric events [15]. However, the timing of SRE onset has not been clarified, and no studies thus far have discussed this. If the timing of SRE onset during VAR smoking cessation treatment could be ascertained, then SRE could be managed, which will be critical for the maintenance of smoking cessation.

Therefore, we surveyed SRE onset timing during VAR smoking cessation treatment using cases reported in the FAERS database. We also surveyed the onset timing for other central nervous system adverse events such as nausea (Nau) and abnormal dreams (including nightmares, ABDs) [16,17], which occur frequently during VAR use, to investigate any characteristics specific to SRE onset timing in reported cases.

Methods

Data sources

We used the FAERS database Quarterly Data Files (Q1 2004 to Q1 2016) published by the FDA (downloaded in September 2016). The Quarterly Data Files comprise seven types of datasets (patient demographic and administrative information, DEMO; drug/biologic information, DRUG; adverse events, REAC; patient outcomes, OUTC; report sources, RPSR; drug therapy start and end dates, THER; and indication for use/diagnosis, INDI). Of these, we used DEMO, DRUG, REAC, and THER files for analysis.

Definition of SRE

The following events were summarised as suicide related events (SRE), taken from the Medical Dictionary for Regulatory Activities (MedDRA, version 19.1) at the Preferred Term level: completed suicide (MedDRA code 10010144), suicidal ideation (10042458), suicide attempt (10042464), suicidal behaviour (10065604), self-injurious ideation (10051154), self-injurious behaviour (10063495), depression suicidal (10012397), intentional self-injury (10022524), poisoning deliberate (10036000), intentional overdose (10022523), and suicide threat (10077417).

Analysis procedures

The procedures, from accessing the Quarterly Data Files to the creation of a time series histogram for SRE reported case numbers, are shown in the flowchart in Figure 1 and described below.

STEP 1. Data extraction

We extracted only cases involving VAR usage from the complete set of Quarterly Data Files. We then extracted cases where SRE had been reported as an adverse event from the subset of VAR usage cases.

STEP 2. Calculating SRE onset time

For VAR usage cases that also reported SREs (SRE+/VAR+ cases), the number of days until SRE onset was calculated by subtracting the “VAR start date” from the “SRE onset date”, for cases where complete descriptions of both of these parameters were available.

STEP 3. Correcting for VAR dosage and administration

According to FDA guidelines [16], VAR dosage and administration should be increased gradually from the start of use (Days 1-3: 0.5 mg/dose once a day; Days 4-7: 0.5 mg/dose twice a day; Day 8 onwards: 1.0 mg/dose twice a day). Among the SRE+/VAR+ cases reported to FAERS, there were cases where SRE onset occurred in less than 7 days although the reported dosage and administration was "1.0 mg/dose twice a day (high-dose)". Such cases may have reported the high-dose VAR start date (Day 8) as START_DT, and an accurate time to SRE may not have been calculated. Thus, in order to evaluate SRE onset timing in VAR+ cases, we set the minimum onset (without correction on the number of days) and the maximum onset (with correction on the number of days), and calculated the number of days to SRE onset in two ways: one with correction (e.g. adding 7 to the number of days to SRE onset in high-dose VAR cases) and the other without correction.

 Figure 1 

Flowchart for time series histogram creation methodology. VAR: varenicline; SRE: suicide related events; Nau: nausea; ABDs: abnormal dreams

Int J Med Sci Image

When the number of days to SRE onset was corrected (i.e. the maximum onset), we divided the dosage and administration reported for each case into six patterns (A: 0.5 mg/dose once a day, B: 0.5 mg/dose twice a day, C: 1.0 mg/dose twice a day, D: Starting Month Pak, E: Continuing Month Pak (Day 29-), F: unclear or unknown), and corrected the number of days to SRE onset for each case based on the pattern. The correction methods were as follows:

If the dosage and administration description pattern for a certain case was A or D, then no correction was made. For pattern B, 3 days were added to the number of days until SRE onset, for pattern C, 7 days were added, and for pattern E, 28 days were added. Any cases with pattern F were excluded from analysis.

When no correction was applied to the number of days to SRE onset (i.e. the minimum onset), we did not correct the number of days to SRE onset in cases showing patterns A to E. Cases exhibiting pattern F were excluded from the analysis.

STEP 4. Eliminating comorbidity effects on suicide risk

Psychiatric disorders such as depression and schizophrenia are already known as risk factors for suicide. Therefore, we excluded cases featuring concomitant antidepressant and antipsychotic agent usage from the SRE+/VAR+ cases [18,19]. In addition, to eliminate the effect of other smoking cessation aids, cases featuring the concurrent administration of other smoking cessation aids were excluded.

STEP 5. De-duplication

The removal of duplicated cases was implemented by referencing the literature on existing reports [20]. First, cases were removed where there was duplication of all items including CASE ID. Next, cases were removed where there were matching demographic data items, excluding the CASE ID.

STEP 6. Creating histograms and removing smoking history effects from the SRE report

After duplicated cases were removed, we created a time series histogram of the number of reported SRE cases per week, including cases where the adjusted number of days until SRE onset presented a negative value (Figure 1).

Factors affecting SRE reports included not only “smoking cessation treatment with VAR”, but also “smoking history” [21]. Therefore, we focused on cases where the number of days until SRE onset presented a negative value, namely, cases where SRE was reported prior to VAR treatment commencement. These cases were not affected by “smoking cessation treatment with VAR” and reported SREs without the influence of diseases that increased suicide risk, such as depression and schizophrenia, due to our previously outlined exclusion criteria. Therefore, we thought that these cases would be affected by “smoking history”. However, if the subset of cases that reported SRE prior to the start of VAR treatment was investigated, it would be possible to eliminate the influence of “smoking history”.

We calculated the “mean number of cases of SRE reported per week +3.0 x standard deviation (SD)” for cases where the number of days until SRE onset was less than 0, referencing the Xbar and R chart concepts [22]. If the number of reported SRE cases in any week after Week 1 exceeded this value, it was determined that SRE incidence was affected by smoking cessation treatment with VAR.

A time series histogram was created for the number of reported cases for each adverse event relating to Nau (MedDRA code 10028813) and ABDs (10000125, 10029412) in cases of VAR usage after data was extracted and processed following the same procedures as that used with SRE, thus also removing the influence of smoking history.

Results

SRE onset time shift in SRE+/VAR+ cases

The number of SRE+/VAR+ cases per week is shown in Figure 2. The data from Week -4 to Week -1 were related to cases where SRE was reported prior to start of VAR usage, and the data from Week 1 onwards were obtained from cases where SRE was reported after the start of VAR treatment.

When we corrected the number of days to SRE onset (i.e. the maximum onset), Week -4 to Week 12 included 689 SRE+/VAR+ cases. The mean number of cases of SRE reported per week and 3.0 x SD from Week -4 to Week -1 were 15.0 cases and 15.3 cases, respectively (mean +3.0 x SD = 30.3). Thus, the decimal points were rounded up, and weeks with the number of reported SRE cases exceeding 31 after Week 1 were marked with * to denote weeks affected by smoking cessation treatment with VAR (Figure 2A). There were a total of 9 weeks (Weeks 1 to 6 and Weeks 8 to 10) in which the number of reported SRE cases exceeded 31.

When we did not correct the number of days to SRE onset (i.e. the minimum onset), 695 SRE+/VAR+ cases were included. The mean number of cases of SRE reported per week and 3.0 x SD from Week -4 to Week -1 were 17.0 cases and 20.0 cases, respectively. Thus, weeks with the number of reported SRE cases exceeding 38 after Week 1 were marked with * (Figure 2B). There were a total of 5 weeks (Weeks 1 to 3, Week 5, and Week 7) in which the number of reported SRE cases exceeded 38.

Nau and ABDs onset time shift in VAR+ cases

The number of VAR+ cases each week that reported on Nau (Nau+/VAR+ cases) is shown in Figure 3. The data from Week -4 to Week -1 were obtained from cases that reported Nau prior to start of VAR treatment, and cases that reported Nau after start of VAR treatment were noted Week 1 onwards.

In the maximum onset, Week -4 to Week 12 included 954 Nau+/VAR+ cases. The mean number of cases of Nau reported per week from Week -4 to Week -1 was 27.3 cases, and 3.0 x SD was 38.3 cases. Thus, weeks with the number of reported Nau cases exceeding 66 after Week 1 were marked with * (Figure 3A). In each week from Week 1 to Week 3, the number of reported Nau cases exceeded 66.

 Figure 2 

Number of SRE reports per week in cases undergoing smoking cessation treatment with VAR. Week -4 to Week -1 includes cases that reported SRE before starting smoking cessation treatment with VAR, Week 1 to Week 12 includes cases that reported SRE after starting smoking cessation treatment with VAR. * Indicates periods affected by smoking cessation treatment with VAR (A; ≥ 31 cases/week, B; ≥ 38 cases/week). VAR varenicline; SRE: suicide related events

Int J Med Sci Image
 Figure 3 

Number of Nau reports per week in cases undergoing smoking cessation treatment with VAR. Week -4 to Week -1 includes cases that reported SRE before starting smoking cessation treatment with VAR, Week 1 to Week 12 includes cases that reported Nau after starting smoking cessation treatment with VAR. * Indicates periods affected by smoking cessation treatment with VAR (A; ≥ 66 cases/week, B; ≥ 79 cases/week). VAR varenicline; Nau: nausea

Int J Med Sci Image
 Figure 4 

Number of ABD reports per week in cases undergoing smoking cessation treatment with VAR. Week -4 to Week -1 includes cases that reported SRE before starting smoking cessation treatment with VAR, Week 1 to Week 12 includes cases that reported ABDs after starting smoking cessation treatment with VAR. * Indicates periods affected by smoking cessation treatment with VAR (A; ≥ 47 cases/week, B; ≥ 59 cases/week). VAR varenicline; ABDs: abnormal dreams

Int J Med Sci Image

In the minimum onset, Week -4 to Week 12 included 952 Nau+/VAR+ cases. The mean number of cases of Nau reported per week from Week -4 to Week -1 was 31.0 cases, and 3.0 x SD was 47.4 cases. Thus, weeks with the number of reported Nau cases exceeding 79 after Week 1 were marked with * (Figure 3B). In each week from Week 1 to Week 2, the number of reported Nau cases exceeded 79.

The number of VAR+ cases each week reporting ABDs (ABDs+/VAR+ cases) is shown in Figure 4. In the maximum onset, Week -4 to Week 12 included 729 ABDs+/VAR+ cases. The mean number of cases of ABDs reported per week from Week -4 to Week -1 was 21.5 cases, and 3.0 x SD was 25.0 cases. Thus, weeks with the number of reported ABDs cases exceeding 47 after Week 1 were marked with * (Figure 4A). In each week from Week 1 to Week 4, the number of reported ABDs cases exceeded 47.

In the minimum onset, Week -4 to Week 12 included 731 ABDs+/VAR+ cases. The mean number of cases of ABDs reported per week from Week -4 to Week -1 was 23.8 cases, and 3.0 x SD was 34.8 cases. Thus, weeks with the number of reported ABDs cases exceeding 59 after Week 1 were marked with * (Figure 4B). In each week from Week 1 to Week 2, the number of reported ABDs cases exceeded 59.

Discussion

Study method validity (removal of smoking history influence)

The aim of this study was to investigate SRE onset timing during smoking cessation treatment with VAR using cases reported in the FAERS database, and to ascertain whether the characteristics of SRE onset timing are different from those of other adverse events commonly seen in cases using VAR.

In the maximum onset, nausea onset signals due to VAR use were detected only in the period from Week 1 to Week 3. In the minimum onset, these signals were detected only in the period from Week 1 to Week 2. Thus, it was considered that nausea due to VAR use was expressed only during the initial administration. Furthermore, smoking cessation treatment with VAR was considered a factor for nausea induction in the initial VAR administration period (Figure 3).

According to the interview form [23], the incidence of nausea was concentrated in the 1 to 2 week period after starting VAR usage, followed by a decline. Therefore, our study showed extremely similar results to those found in the clinical trials. This validates our decision to remove the influence of “smoking history”, making it possible to evaluate the effect of VAR treatment on each individual adverse event (SRE, nausea, and abnormal dreams).

SRE onset timing characteristics during VAR treatment

In the maximum onset, SRE onset affected not by smoking history but by smoking cessation treatment with VAR was observed in two periods: Weeks 1 to 6 and Weeks 8 to 10. This suggests that smoking cessation treatment with VAR may cause SRE during 9 weeks out of the 12 week administration period of VAR. However, in the minimum onset, it was suggested that smoking cessation treatment with VAR may cause SRE during 5 weeks out of 12 week administration period of VAR (Weeks 1 to 3, Week 5, and Week 7).

In the maximum onset, the onset of nausea and abnormal dreams, which are central nervous system adverse events frequently observed during smoking cessation treatment with VAR, was observed only during the period from Weeks 1 to 3 and Weeks 1 to 4, respectively (Figures 3A, 4A). In the minimum onset, the onset of both nausea and abnormal dreams was observed only in the period from Week 1 to Week 2 (Figures 3B, 4B). It was suggested that the onset of nausea and abnormal dreams induced during VAR administration was observed only during the initial administration. It is almost consistent with the onset observed in clinical trials [23].

Therefore, it was found that SRE observed during smoking cessation treatment with VAR showed a different expression profile than that of nausea and abnormal dreams, which are also central nervous system adverse events. Although the signal detection period of SRE affected by smoking cessation treatment with VAR varied depending on the presence or absence of day correction, SRE was expressed over a long period from the start of VAR administration regardless of the presence or absence of the correction. Thus, it was suggested that caution against SRE onset should always be required.

Abbreviations

ABDs: abnormal dreams; FAERS: FDA Adverse Event Reporting System; FDA: Food and Drug Administration; MedDRA: the Medical Dictionary for Regulatory Activities; Nau: nausea; SD: standard deviation; SRE: suicide related events; VAR: Varenicline.

Competing interests

The authors have no competing interests to declare.

References

1. Tobacco Fact sheet. Updated June 2016 WHO. http://www.who.int/mediacentre/factsheets/fs339/en/

2. Institute for Economics and Policy. 2010 A study on the role of non-smoking policy. http://www.ihep.jp/publications/report/search.php?dl=26&i=1

3. The health consequences of smoking - 50 years of progress: a report of the Surgeon General, 2014. U.S. Department of Health and Human Services. http://www.surgeongeneral.gov/library/reports/50-years-of-progress/index.html

4. Xu X, Bishop EE, Kennedy SM. et al. Annual healthcare spending attributable to cigarette smoking: an update. Am J Prev Med. 2015;48(3):326-333

5. Fiore MC, Jaen CR, Baker TB. et al. Treating tobacco use and dependence: 2008 update, clinical practice guideline. U.S. Department of Health and Human Services. 2008

6. Gonzales D, Rennard SI, Nides M. et al. Varenicline, an α4β2 nicotinic acetylcholine receptor partial agonist, vs sustained-release bupropion and placebo for smoking cessation: a randomized controlled trial. JAMA. 2006;296:47-55

7. Anthenelli RM, Benowitz NL, West R. et al. Neuropsychiatric safety and efficacy of varenicline, bupropion, and nicotine patch in smokers with and without psychiatric disorders (EAGLES): a double-blind, randomised, placebo-controlled clinical trial. Lancet. 2016;387:2507-2520

8. Aubin H-J, Bobak A, Britton JR. et al. Varenicline versus transdermal nicotine patch for smoking cessation: results from a randomised open-label trial. Thorax. 2008;63:717-724

9. FDA requires new boxed warnings for the smoking cessation drugs Chantix and Zyban. http://www.fda.gov/Drugs/DrugSafety/DrugSafetyPodcasts/ucm170906.htm

10. Freedman R. Exacerbation of schizophrenia by varenicline. Am J Psych. 2007;164:1269

11. Yousefi MK, Folsom TD, Fatemi SH. A review of varenicline's efficacy and tolerability in smoking cessation studies in subjects with schizophrenia. J Addict Res Ther. 2011;4:3045

12. U.S. Food and Drug Administration. FDA drug safety newsletter, volume 2, number 1, 2009. http://www.fda.gov/Drugs/DrugSafety/DrugSafetyNewsletter/ucm107311.htm

13. CzeekV. https://pro.czeek.com/aers/main/SearchMain.do

14. Thomas KH, Martin RM, Knipe DW. et al. Risk of neuropsychiatric adverse events associated with varenicline: systematic review and meta-analysis. BMJ. 2015;350:h1109

15. Akimoto H, Oshima S, Negishi A. et al. Assessment of the risk of suicide-related events induced by concomitant use of antidepressants in cases of smoking cessation treatment with varenicline and assessment of latent risk by the use of varenicline. PLoS ONE. 2016;11:e0163583

16. the U.S. Food and Drug Administration. Drugs@FDA. http://www.accessdata.fda.gov/scripts/cder/drugsatfda/index.cfm

17. Pfizer. http://www.pfizer.com/news/press-release/press-release-detail/chantix_champix_varenicline_demonstrates_smoking_cessation_efficacy_in_smokers_unwilling_or_unable_to_quit_abruptly

18. Goldston DB, Daniel SS, Erkanli A. et al. Psychiatric diagnoses as contemporaneous risk factors for suicide attempts among adolescents and young adults: Developmental changes. J Consult Clin Psychol. 2009;77:281-290

19. Oquendo MA, Galfalvy H, Russo S. et al. Prospective study of clinical predictors of suicidal acts after a major depressive episode in patients with major depressive disorder or bipolar disorder. Am J Psych. 2004;161:1433-1441

20. Banda JM, Evans L, Vanguri RS. et al. A curated and standardized adverse drug event resource to accelerate drug safety research. Sci Data. 2016;3:160026

21. Zvolensky MJ, Bakhshaie J, Sheffer C. et al. Major depressive disorder and smoking relapse among adults in the United States: A 10-year, prospective investigation. Psychiatry Res. 2015;226:73-77

22. NIST/SEMATECH e-Handbook of Statistical Methods. NIST. http://www.itl.nist.gov/div898/handbook/pmc/section3/pmc321.htm

23. champix® interview form. Ministry of Health, Labour and Welfare. http://www.info.pmda.go.jp/go/interview/2/671450_7990003F1028_2_1F.pdf

Author contact

Corresponding address Corresponding author: Phone & FAX: +81-49-271-7056; E-mail: dkobaac.jp


Received 2017-3-1
Accepted 2017-5-17
Published 2017-7-20


Citation styles

APA
Akimoto, H., Wakiyama, H., Oshima, S., Negishi, A., Ohara, K., Numajiri, S., Okita, M., Ohshima, S., Inoue, N., Kobayashi, D. (2017). Identification and Characteristics of Time-Related Shifts in Suicide-Related Event Frequency During Smoking Cessation Treatment with Varenicline. International Journal of Medical Sciences, 14(10), 920-926. https://doi.org/10.7150/ijms.19877.

ACS
Akimoto, H.; Wakiyama, H.; Oshima, S.; Negishi, A.; Ohara, K.; Numajiri, S.; Okita, M.; Ohshima, S.; Inoue, N.; Kobayashi, D. Identification and Characteristics of Time-Related Shifts in Suicide-Related Event Frequency During Smoking Cessation Treatment with Varenicline. Int. J. Med. Sci. 2017, 14 (10), 920-926. DOI: 10.7150/ijms.19877.

NLM
Akimoto H, Wakiyama H, Oshima S, Negishi A, Ohara K, Numajiri S, Okita M, Ohshima S, Inoue N, Kobayashi D. Identification and Characteristics of Time-Related Shifts in Suicide-Related Event Frequency During Smoking Cessation Treatment with Varenicline. Int J Med Sci 2017; 14(10):920-926. doi:10.7150/ijms.19877. https://www.medsci.org/v14p0920.htm

CSE
Akimoto H, Wakiyama H, Oshima S, Negishi A, Ohara K, Numajiri S, Okita M, Ohshima S, Inoue N, Kobayashi D. 2017. Identification and Characteristics of Time-Related Shifts in Suicide-Related Event Frequency During Smoking Cessation Treatment with Varenicline. Int J Med Sci. 14(10):920-926.

This is an open access article distributed under the terms of the Creative Commons Attribution (CC BY-NC) license (https://creativecommons.org/licenses/by-nc/4.0/). See http://ivyspring.com/terms for full terms and conditions.
Popup Image