Evaluating an online well-being program for college students during the COVID-19 pandemic

Introduction: The global COVID-19 pandemic has aggravated challenges involving college students’ mental health and well-being. Some literature suggested developing online programs to address the pandemic’s impact on college students’ mental health and well-being. Thus, this study assessed if significant improvement in well-being among college students can be observed after introducing an online well-being program. Methods: The study utilized a quantitative methodology, mainly using a two-group pretest-posttest design on 178 college students in a private college and state university. The experimental group received 3 months of the well-being program while the control resumed their activities of daily living (ADL). The modified positive emotion, engagement, relationship, meaning, and accomplishment (PERMA) profiler questionnaire was the primary evaluation instrument that measured the participants’ well-being. The first phase gathered the participants’ relevant profile and background, and the last phase concluded with the evaluation of the program. Data were analyzed using SPSS v.21. Results: Based on the post-evaluation PERMA scores, the experimental participants (M = 7.21, SD 1.70) did not differ much from the control (M = 7.07, SD = 1.55) according to a t-test t(176) = –1.07, p = 0.57 as computed using a two-sample independent t-test at a significance level of α = 0.05. The overall PERMA score description is normal functioning. The Pearson correlation of the experimental group’s pre-test and post-test scores ( r (91) = 0.01, p = 0.904) and the control ( r (83) = 0.04, p = 0.732) group did not indicate an evidence of a significant relationship. Conclusion: The results do not provide evidence of a significant difference and relationship between the experimental participants’ pre-test and post-test PERMA scores after the online well-being program.


INTRODUCTION
COVID-19 was initially identified in Wuhan, China, in 2019 (1) and has spread to over 200 countries (2). In March 2020, the World Health Organization proclaimed a global epidemic (3). Since COVID-19 was announced as a worldwide epidemic, public health has become a concern, particularly mental health. National governments have compelled millions of people, including educators, corporate executives, and students, to remain protected by self-isolation or by enforcing a full or partial global lockdown (4). Access to the classroom has been restricted due to the long-standing lockout. Around 1.5 billion schoolaged children and university students have been forced to withdraw from their institutions due to the COVID-19  (5). Online classes are in high demand as a viable alternative to institutional closure in this unprecedented period. On the other hand, students and teachers confront a variety of impediments and difficulties, including psychological problems produced by an insufficient learning method (6).
Another set of academic difficulties stemmed from the abrupt transition to remote teaching among higher education institutions. College students were subject to the teaching decisions made by each faculty member. The faculty has chosen to teach synchronously or asynchronously through various technological platforms. Due to a lack of planning time for this new delivery method, faculty members may have overlooked some students' accessibility and accommodation needs, putting visually impaired students, those with hearing difficulties, and others at a disadvantage. In more normal circumstances, these academic difficulties may result in students lacking independence and an unexpected inability to learn material they could easily comprehend.
Another issue that faculty members and students face during stay-at-home orders is loneliness, resulting in a lack of loving feelings and belonging (7). While we communicate electronically with colleagues and students throughout the day, the virtual world provides only a limited amount of artificial human interaction, resulting in a sort of worry called "Zoom fatigue." This simplified representation is associated with an increase in mood swings, whereas increasing self-complexity acts as a buffer against the adverse effects of stressful events. Before the pandemic of COVID-19, more than 35% of 5-year college students reported having been diagnosed with anxiety, mood, or substance use disorder (8).
According to recent research on virus epidemics and pandemics, stressors included fears of infection, anger, boredom, a lack of supplies and knowledge, financial loss, and stigma (9). A recent study on COVID-19's psychological effects has primarily been conducted in China's first hotspots. While some research explored mental health issues during epidemics, the majority is primarily on health care workers, patients, newborns, and the general community (10). The Kaiser Family Foundation survey reported negative mental health effects from COVID-19-related anxiety or stress (11). In general population samples, North America and Europe also discovered increased anxiety and depressive symptoms (12), (13). Except for a few reports from China (14), there is no evidence of the present pandemic's mental health impact on the vulnerable population of college students (15). While these studies indicate a rise in mental health issues among college students, the underlying variables may be inapplicable to populations in other nations. Thus, as various recent studies have stressed, there is a priority to evaluate the impact of the rising epidemic on the mental health and well-being of college students (11).
College students, in particular, have struggled to adapt to an entirely virtual world, compounded by heavy class loads and the transition to more independent adult lives (16). Significant traumatic and stressful incidents, such as those associated with the COVID-19 pandemic, may exacerbate the strains on college students who are still developing their identities (17). Certain students may be confronted with unmet physiological, psychological, and safety requirements, most likely in areas where they have not previously encountered deficiency. When these needs are not met, students cannot focus on their studies (18). Some literature suggested developing online programs to address the pandemic's impact on college students' mental health and well-being (19), (20). To address this gap, we developed an online well-being program based on cognitive behavioral therapy (CBT), analysis, design, development, implementation, and evaluation (ADDIE) model anchored on the positive emotion, engagement, relationships, meaning, and accomplishments (PERMA) construct. As such, this study examined whether a significant improvement in well-being can be observed among college students following the implementation of an online well-being program.

METHODS
The study assessed if significant well-being improvement can be observed among college students at a private college and state university undergoing an online well-being program. It presents the sociodemographic profile of the participants in terms of age, gender, and socioeconomic levels; the pre-test and post-test scores for both the control and experimental groups' well-being in terms of positive emotion, engagement, relationship, meaning, and accomplishment (PERMA) scores; the investigation of the differences and relationships between the pre-test and post-test PERMA scores of the control and experimental groups; and to determine if the online well-being program improved the well-being of the participants in the experimental group.
This study used a pretest-posttest control group design (21). The data collection procedures, informed consent forms, and data collection instruments were given ethical approval by the University Research Ethics Committee (UREC) of the Cebu Technological University with UREC Protocol Number: 001-2021. Data on the participants' sociodemographic profile and well-being were gathered from the experimental and control groups before the online well-being program. The program was provided only to the experimental group while the control resumed their activities of daily living (ADL). The participants' well-being was measured using the PERMA questionnaire. After the conduct of the program, the participants' well-being as post-test data was gathered again from the experimental and control groups using the same instrument. Due to the nature of the study, wherein the researchers need to be visible in the virtual learning environment, the allocation was not blinded to those delivering the intervention.
The participants were college students enrolled in a private college and state university for the second semester of the academic year 2020-2021. These participants were the 1 st year to 3 rd year students taking up nursing, criminology, accountancy, elementary education, secondary education (major in TLE and English), hospitality management, industrial engineering, fisheries, and industrial technology (major in computer technology, automotive, welding, and drafting).
The sample size of each experimental and control group was determined using G*Power3. 1.9.7 (22) setting to Cohen's medium effect size of 0.5, significance level 0.05, and statistical power of 0.80. Six hundred students were assessed for eligibility. Four hundred and seventy were excluded due to not meeting inclusion criteria (n=180) and declined to participate in the study (n=290). A minimum sample size was computed. The minimum calculated sample size is 130. Due to more student participation, the total sample size reached 178, with 93 allocated for the experimental and control groups. However, eight participants from the control group lost follow-up because they did not respond when contacted by the researchers (Figure 1).
The study used a convenience sampling method (21). Although the true experimental design is considered a more rigorous test of intervention outcomes (23), there was a minor error in the randomization process, which the researchers later found out.
The study was conducted for a period of 6 months, from May 2021 to October 2021.
The research participants included in this study were enrolled at a private college and state university for the second semester of 2020-2021, from the 1 st year to 3 rd year.
Based on their responses to the online questionnaire, they must not have any ailments such as high blood pressure, diabetes mellitus, or a history of diagnosed mental health or behavioral disorders. In addition, graduating students, unable to attend at least two sessions, and unwillingness to continue participating in the research were excluded.
The ethical clearance was obtained from the University Research Ethics Committee (UREC), Cebu Technological University, Main Campus, Cebu City, Philippines, with UREC Protocol Number: 001-2021. After approval, the participants' informed consent was included in the study. They were oriented on the flow and the duration of the study.
The online validated instrument utilized for the study is the modified PERMA (positive emotion, engagement, relationships, meaning, and accomplishment) Profiler Questionnaire by Martin Seligman (24) to determine the well-being of the participants. The online instrument was prepared using Google Forms  and the same instrument was distributed to the experimental and control participants during Phase I (orientation) and Phase IV (evaluation) of the study.
The online instrument used for the face validation of experts pre-implementation was the same instrument utilized in the post-evaluation among the experimental participants. It assessed if the well-being program topics are of interest to the participants, the program's content is relevant, and if the information added to the current body of knowledge.
The modified PERMA Profiler questionnaire was designed using a rating scale. Answers were based on an 11-point scale (completely/always = 10 and not at all/never = 0). The statements were based on the following constructs: Positive emotion, engagement, relationships, meaning, and accomplishment. A pilot study was conducted on 30 students excluded from the survey to ensure clarity of the questionnaires, employing coherence and consistency tests. The instrument was translated forward and backward. A language specialist translated the scales' English version into Cebuano (a local dialect in the Philippines) and then back into English by another expert. The translated items were finalized by a psychiatric nurse, a language expert, and two university professors. The questionnaire was developed in response to student feedback.
In the study conducted by Wammerl et al. in 2019, the modified PERMA Profiler questionnaire demonstrates high concept validity, factorial and convergent validity, adequate reliability, and the first signs of measurement invariance gender and nationality (25). The questionnaire's reliability was assessed using Cronbach's alpha values ranging from 0.60 to 0.95 for the primary PERMA factors based on the study conducted by Pezirkianidis et al. in 2019 (26). In test-retest analyses, Pearson r values in confirmatory factor analysis ranged from 0.53 to 0.90. The fit indices indicated that the model was well fitted (factor loadings ranged from 0.36 to 0.93, and the correlations between the factors were in the predicted direction, indicating that the model had good convergent and divergent validity) as anchored on the study by Umucu et al., 2020 (27).
The validation instrument used in the study was adapted from Arora (28) on the Development and Validation of Health Education Tools and Evaluation Questionnaires. The program was submitted for examination to three experts (one university professor, one psychiatrist, and one   capacity. 3. The materials were produced up to the task of what they were intended. Implementation To ensure that the participants were doing/applying the CBT (cognitive behavioral therapy)-based interventions, the P-PIE (Problem-Plan, Intervention, and Evaluation) Self-Assessment Form© was used. The P-PIE (Problem-Plan, Intervention, and Evaluation) Self-Assessment Form© was a comprehensive reflection tool that assisted the participants in applying the study's findings to their own context. The participants used this document to capture their insights from the session activities, subjective assessments, and reflection questions throughout the study. Moreover, the participants discovered that the P-PIE (Problem-Plan, Intervention, and Evaluation) Self-Assessment Form served as a lasting record of their learning and ideas for improving their physical health and well-being after the study. Evaluation To evaluate the program's effectiveness, participants completed a post-study questionnaire and an evaluation instrument that included the following parameters: The participants are interested in the topic; the content is relevant; and the information added to current knowledge. In addition, members of the experimental group were interviewed online about their experiences with the program.
psychiatric nurse). The write-up of the online well-being program was first presented to the university professor for improvement and possible correction based on whether or not the topic is of interest to the college students, the relevance of the content, and if the information can add to the existing knowledge. The improved write-up was then submitted to the psychiatrist and the psychiatric nurse for more improvements. A series of online meetings took place to discuss their concerns, followed by compliance with their suggestions. Discussions of three experts and researchers focus on the content to be presented as inputs to the participants and the questions asked in the questionnaire. After implementing the recommendations, these three experts approved the online well-being program for implementation.
The validation result by the three experts is presented in Table 1.
The computed mean for the topics are of interest to the participants got an overall rating of 3.00 (agree), relevance with 3.33 (strongly agree), and the information added to existing knowledge with a rating of 3.00 (agree). The overall rating of the three validation parameters got an overall rating of 3.11 (agree). According to Barrable and Shackleton (20), (29), developing a well-being program must be anchored on evidence-based guidelines to benefit end-users.
In addition, the well-being program was also validated using the ADDIE (analysis, design, development, implement, and evaluation) model (30)(31)(32), as presented in Table 2. The well-being program's objectives were to provide students with an online community and opportunities for interaction, introduce students to fundamental concepts of health and well-being, promote mental health, and equip students with tools to improve their well-being. These objectives are consistent with the PERMA framework for creating a pleasant and meaningful life and contribute to developing a health education approach by teaching well-being skills in conjunction with academic courses (24).
Before the program started, the participants were informed that participation in the study was entirely voluntary and free from any undue influence. They were also given enough time to decide whether to participate or not. Should they have any questions, they were instructed to contact the lead researcher through email at any time. In addition, they were informed that the data gathered would only be for research purposes.
The seven sessions of the well-being education (Phase II) were delivered by the principal researcher assisted by the coresearchers through Zoom  . The well-being program was based on cognitive behavioral therapy (33), (34) techniques, anchored on the PERMA (positive emotions, engagement, relationships, meaning, and accomplishments) construct (24). The duration was between 45 and 60 minutes -one session per week. The schedule was arranged based on the agreed date and time of the participants. Participants' well-being is normal. 5.00-6. 40 Suboptimal functioning (SF) Participants' well-being is below the optimal (best possible) level. 0.00-5.00 Languishing (L) Participants' well-being is failing to make progress.
During educational sessions, teaching methods such as focus group discussions and dialogue sessions were done through Zoom  , PDF pamphlets, and PowerPoint  Slides, designed based on the CBT (cognitive behavioral therapy) which are provided.
To ensure that the participants were doing/applying the CBT (cognitive behavioral therapy)-based interventions, the P-PIE (Problem-Plan, Intervention, and Evaluation) Self-Assessment Form© was used. The P-PIE (Problem-Plan, Intervention, and Evaluation) Self-Assessment Form© was a comprehensive reflection tool that assisted the participants in applying the study's findings to their own context. The participants used this document to capture their insights from the session activities, subjective assessments, and reflection questions throughout the study. Moreover, the participants discovered that the P-PIE (Problem-Plan, Intervention, and Evaluation) Self-Assessment Form served as a lasting record of their learning and ideas for improving their physical health and well-being after the study.
Given the instrument's online nature, the researchers ensured participants' anonymity and saved and secured data collected with the assistance of the MIS and computer technology department. Following that, the results were retrieved, tabulated, computed, and interpreted to produce findings, conclusions, and recommendations.
Presented in Table 3 is the scoring procedure anchored on the positive emotion, engagement, relationships, meaning, and accomplishments (PERMA) model. Since measuring well-being is complex, there are no precise cutoffs regarding low functioning, good functioning, and flourishing (35). However, it could be based on the validation studies (35), (36). Well-being measures usually are skewed toward the positive, such that the midpoint becomes about 6.5-7.5 (not 5, the middle of the 0-10 scale).
The sociodemographic profile of the participants was expressed as frequencies, percentages, and means. The Chi-square test of independence was utilized to compute if a significant association exists between variables. Two-sample paired and independent t-tests were used to assess the significance of the difference between the experimental and control groups in the pre-test and post-test PERMA scores. Pearson correlation was used to determine if a significant relationship exists between the pre-test and post-test scores of the experimental and control groups. The significance level of all tests of difference and relationship is set at α=0.05. All analyses were performed using SPSS v.21.

RESULTS
The distribution of research participants with the computed Chi-square results is presented in Table 4.
The majority of the research participants of this study were 20 (21.51%) and 21 (31.18%) years old. About 76.40% of these participants were females. Finally, the majority, 43.82%, of the participants belong to the 5-6 socioeconomic level, which means they belong to the middle-income group (37). These are families earning between 2 and 12 times the poverty level. In terms of age, the computed Chi-square statistic is 0.61 with p = 0.96. Furthermore, the calculated Chi-square for gender is 0.03 with p = 0.85. Finally, the socioeconomic level had a Chi-square statistic of 0.85 with p = 0.93. The Chi-square test of independence showed no significant association between the experimental and control groups' age, gender, and socioeconomic level.
The experimental and control groups' pre-test and post-test means with the corresponding sample standard deviation values in each PERMA factor and the computed p-values can be gleaned in Table 5.
The results imply that the research participants in the experimental did not differ much from the control group in their ratings in each PERMA factor and the overall PERMA in both the pre-test and the post-test with an overall PERMA score description of normal functioning as computed using a two-sample paired t-test at a significance level of α = 0.05.
The experimental and control group's overall pre-test and post-test means are shown in Table 6 with corresponding sample standard deviation values of the PERMA scores.
The experimental group's overall mean at the pre-test was 7.07 (SD = 1.44), while it increased slightly to 7.21 (SD = 1.47) in the post-test with a description of normal functioning. The increase is not significant since the t-test = -1.07 (p = 0.29). The overall mean for the control group in the pre-test was 7.30 (SD = 1.47), while it decreased slightly to 7.07 (SD = 1.55) in the post-test with a description of normal functioning. The decrease is not  significant since the t-test = 0.57 (p = 0.57). Moreover, the calculated effect size for the pre-test difference between the experimental and control groups is at Cohen's d = 0.34. and the post-test difference between the two groups has a computed effect size of Cohen's d = 0.09 indicating a small effect size.
Pearson correlation coefficient was computed to assess the linear relationship between the experimental and control group's pre-test and post-test PERMA scores. The results of the Pearson correlation of the experimental group (r(91) = 0.01, p = 0.904) and the control (r(83) = 0.04, p = 0.732) do not allow a conclusion of a significant relationship between the pre-test and post-test results.
The experimental participants (N = 93) completed a post-study evaluation instrument to evaluate the online well-being program. It was the same tool used during the face validation of the study. To review, the evaluation instrument assessed if the well-being program topics are of interest to the participants, the program's content is relevant, and if the information added to the current body of knowledge. In addition, participants were interviewed online about their experiences with the program.
As can be gleaned in Table 7, 44 (47.31%) of the participants strongly agree that the topics of the well-being program are interesting. Meanwhile, 15 (16.13%) of the participants strongly disagree. In terms of relevance, 46 (49.46%) of the participants rated the program strongly agree and 12 (12.90%) rated strongly disagree.
Information added to the current body of knowledge was rated strongly agree among 43 (46.24%) participants, and on the contrary, 10 (10.75%) participants rated strongly disagree.
Based on an online interview conducted after the post-evaluation survey, the experimental participants verbalized that the online well-being program is timely and relevant during this COVID-19 pandemic. However, they believe that the well-being program can be beneficial even without the pandemic.
Furthermore, although the well-being program platform is through a virtual learning environment, the discussion was interactive. One participant suggested conducting the online well-being program in a face-to-face setting because she believes that the participants can be more engaged in the aforementioned platform.

DISCUSSION
This study assesses if a significant improvement among college students' well-being in terms of positive emotion, engagement, relationships, meaning, and accomplishment (PERMA) construct. Based on the post-evaluation PERMA scores, the experimental participants (M = 7.21, SD 1.70) do not differ much from the control (M = 7.07, SD = 1.55) according to a t-test t(176) = -1.07, p = 0.57 as computed using a two-sample independent t-test at a significance level of α = 0.05. The overall PERMA score description is normal functioning. The Pearson correlation of the experimental group's pre-test and posttest scores (r(91) = 0.01, p = 0.904) and the control (r(83) = 0.04, p = 0.732) group did not indicate an evidence of significant relationship Although our results showed no significant difference and relationship in the pre-test and post-test PERMA scores among the experimental participants, still, well-being programs are essential among students. Since well-being has many facets, improving students' well-being requires a whole-school approach involving teachers and parents. Moreover, institutions should provide lessons focused on adopting a healthy lifestyle and how to prevent or cope with health problems in collaboration with those involved, including health and social services, local authorities, and civil society organizations (38), (39).
Internet-based interventions to promote well-being have been supported in higher education. Online programs are more feasible than face-to-face or blended delivery techniques because of ease of access and interaction flexibility (19). In addition, online programs benefit those who avoid seeking help due to stigma, making them accessible to individuals who may not have sought care in the past (20). A systematic review demonstrated the efficacy of internet-based programs in addressing mental health issues common among college students and in enhancing overall well-being (40). It is critical to remember that internet delivery was the only option in the early stages of the pandemic due to government constraints. As a result, developing an online well-being program was both practical and feasible. In terms of program participation during the pandemic, our research indicates that face-to-face well-being programs may have prompted more active participation and integration with peers (19). This demonstrates the critical nature of balancing accessibility and engagement in future program delivery.
The virtual learning environment was used to conduct the well-being program. The virtual learning environment was critical in enabling students to participate in weekly well-being sessions and expanded access to well-being assistance sources through the Amoma Project social media group. Having permanent access to materials was critical to the program's success since self-management of well-being and mental health is vital for persons who are hesitant to seek assistance (41). This method aimed to foster autonomy, self-advocacy for well-being, and similar online interventions (42).
There is an argument that well-being programs could help retain college students, specifically as students often withdraw from university due to personal reasons, including mental health (19). Thus, prioritizing student mental health and well-being should benefit both mental health outcomes and future research on the long-term benefits of well-being interventions on student retention.
Our well-being program follows the ADDIE (analysis, design, development, implement, and evaluation) model, integrated into the educational contexts (30), (32). A more practical method of fostering well-being is incorporating character and well-being development within the curriculum (43). Morgan et al. (19) have contended a "whole-university" approach to addressing well-being programs. Furthermore, Morgan and Simmons (19) recommended that well-being programs be incorporated into the curriculum to better understand the different concepts of well-being within the curriculum processes to foster inclusive practices and a sense of belonging. This is not an easy endeavor, as it necessitates a concerted effort across all higher education institutions. Similarly, collective action-based, whole-school approaches to mental health and well-being appear to be more effective while accounting for the complexities of educational systems (19). A holistic approach to well-being that incorporates and embeds it across the university's processes will likely be more beneficial than stand-alone initiatives in higher education institutions. We hope that the higher education institutions will integrate mental health and well-being frameworks throughout the various levels integrated into the curricula and well-established core requirements.
The present study used a limited sample size, and it is necessary to replicate the findings using a larger sample size. The outcome variables were assessed using self-reporting techniques. In future studies, including objective metrics will help to increase the research validity. Despite these limitations, this study contributes to the existing body of knowledge by developing an online well-being program for college students during the COVID-19 pandemic.

CONCLUSION
The results do not provide evidence of a significant difference and relationship between the experimental participants' pre-test and post-test PERMA scores after the online well-being program. The result of this study is relevant for individuals without physical or mental debilitating diseases. A longitudinal study utilizing a face-to-face approach may be undertaken to determine if there is a significant difference with the virtual learning environment.