Adaptation of the incivility in nursing education-revised survey: a validity and reliability study

Abstract

Background

Academic incivility is one of the most significant issues that disrupt the learning process for nursing students and adversely affects the quality of nursing care. Therefore, it is crucial to assess incivility within the nursing academic environment using valid and reliable measurement tools. The aim of this study is to translate and adapt the Incivility in Nursing Education-Revised (INE-R) Survey into Turkish and assess its validity and reliability for nursing students and faculty.

Methods

The cross-cultural adaptation and validation of the INE-R were carried out within the framework of a methodological cross-sectional study. The Turkish validity and reliability of the INE-R survey, which assesses the behaviors of faculty incivility, were examined with 522 nursing students from 11 different universities, including students from the 2nd, 3rd, 4th year. Data were collected from April 20, 2021 to October 30, 2021.

Results

Following the cross-cultural adaptation, the confirmatory factor analysis (CFA) confirmed that the Turkish version of the INE-R, with 24 items and a two-factor structure, showed a good fit. The reliability analysis revealed Cronbach’s alpha of 0.952 for the total score, 0.899 for the low-level incivility subscale, and 0.991 for the high-level incivility subscale. Item analysis indicated that all items had item-total correlation values above 0.30, with none removed. Additionally, removing any item did not improve the scale’s Cronbach’s alpha. The test-retest results showed no statistically significant differences.

Conclusion

The Turkish adaptation of the INE-R demonstrates its validity and reliability as a valuable instrument for assessing the perceptions of Turkish nursing students regarding uncivil faculty behaviors, as well as the frequency of their exposure to such behaviors.

Impact statement

INE-R survey has demonstrated psychometric reliability and validity, making it a valuable tool for assessing incivility within nursing academia.

Introduction

It is stated that more than half of nurses have witnessed or experienced incivility [1]. Nursing students and faculty members also face incivility in the academic nursing environment [2]. Recent studies show that nursing students report a wide range of uncivil behaviors from faculty [3], while faculty members also report experiencing uncivil behaviors from students [4]. Studies focus on uncivil behaviors demonstrated by both groups [5, 6].

Academic incivility stands as one of the most significant challenges disrupting the learning process for nursing students and adversely affecting the quality of nursing care [7]. Uncivil behaviors have been shown to significantly hinder the learning experience [8]. Such behaviors can diminish students’ self-esteem, self-efficacy and confidence, thereby creating substantial barriers to their professional development [9, 10]. Additionally, they can have a negative impact on students’ morale, diminishing their motivation and learning abilities [11]. The learning experiences of students during clinical practice also can be significantly undermined by encounters with incivility [12]. Furthermore, it has been observed that these uncivil behaviors negatively impact students’ desire to become professional nurses [13].

Background

Bullying, violence, and incivility are significant and ongoing issues that have been discussed in nursing for a long time. The concept of bullying was first defined by Roberts (1983) as a situation in which nurses, due to a lack of power and autonomy, engage in violent behavior toward one another (Condon, 2015). Incivility, on the other hand, emerged in 1999 under the concept of “workplace incivility” (Andersson & Pearson, 1999) and was incorporated into nursing education in the early 2000s (Clark & Springer, 2007). Bullying is defined as mistreatment that causes harm to health (JCI, 2021), while lateral violence is described as acts of incivility occurring between individuals of equal status (Tian et al., 2019). Vertical violence, however, is explained within the context of hierarchical power relations, as acts of violence/incivility directed by superiors toward subordinates (Fudge, 2006; Tian et al., 2019). While bullying and violence are typically described as more serious behaviors that result in physical or psychological harm, incivility stands out as subtler behaviors that may occur unnoticed but still have negative effects on nursing education.

Nursing students and newly graduated nurses in their early years of professional practice are among the most vulnerable groups when it comes to incivility. In addressing the issue of incivility, experienced nursing faculty members should play a leadership role [14, 15]. The measurement of incivility issues in nursing education is essential for the development of strategies and interventions aimed at improving the overall culture in educational and clinical settings, enhancing student learning outcomes, strengthening student-faculty relationships, and supporting the retention rates of students and faculty members in the nursing profession [6, 16].

Exploratory research on uncivil behavior is crucial for nursing faculty to identify at-risk students and intervene effectively. Further studies should focus on defining incivility, understanding contributing factors, and assessing the impact of interventions, thereby providing educators with the tools to address incivility issues [17, 18]. Assessing incivility in the academic nursing environment using valid and reliable measurement tools is crucial [19]. Clark’s INE-R survey is a comprehensive instrument that covers all nursing academic settings, including both classroom and clinical practice, and assesses physical, verbal, non-verbal behaviors, and the use of technology from the perspectives of both faculty members and students [19]. This scale, which has been translated into different languages and culturally adapted [20,21,22], provides ease of use due to its simplicity [22].

This study aims to conduct a Turkish validity and reliability study of the INE-R scale for use in Turkish nursing education. Creating a culture of civility in the nursing academic environment may have a profound impact on the nursing profession as a whole [23].

Research questions

Q1: Is the Turkish version of INE-R valid?

Q2: Is the Turkish version of INE-R reliable?

Research hypothesis

The INE-R will demonstrate strong psychometric properties, including high validity and reliability.

Methods

Aim

Since there is currently no scale available for measuring nursing students’ perceptions and the frequency of incivility in Turkey, this study was undertaken to investigate the psychometric properties of the Turkish adaptation of the Incivility in Nursing Education Revised (INE-R) scale.

Design

The cross-cultural adaptation and validation of the INE-R were carried out within the framework of a methodological cross-sectional study.

Setting and sample

The study group comprised 2nd, 3rd, and 4th-year nursing students (n = 522) enrolled in four-year nursing programs in Turkish universities. Inclusion criteria required students to be proficient in Turkish and willing to participate in the study, and non-random sampling was employed. To ensure heterogeneity, the research was conducted across nursing programs in 11 different universities in Turkey, taking into account the various educational approaches employed by nursing schools (i.e., classical, integrated, internship) and the university type (public and private). The literature highlights the significance of obtaining a sample size exceeding 500 to reveal the factor structure of a test effectively, with fewer than 200 samples potentially being insufficient for psychometric structure examination [24, 25].

Procedures

The research was carried out from April 20, 2021 to October 30, 2021. Given the circumstances of the COVID-19 pandemic, data collection commenced by utilizing Google Forms. A link to the Google Form, which included a demographics questionnaire and the INE-R Scale, was distributed to students through WhatsApp groups. To facilitate student participation, contact was established with the heads of academic departments within the respective schools, and communication was made through student representatives.

As the circumstances related to the COVID-19 pandemic evolved and formal in-person education resumed in September 2021, data collection also shifted to face-to-face interactions, which accounted for 42.1% of the study group.

Measures

Demographics form

The demographics form was developed by the researchers to gather information concerning the socio-demographic characteristics of the students. This included data on age, gender, income, year of study, overall academic performance, primary location of residence, and their motivation for choosing nursing as a field of study.

Incivility in Nursing Education Revised (INE-R)

The INE, initially created by Clark in 2004, stands as the pioneering tool for assessing incivility in nursing education. Over the years, the original scale has been employed in numerous research studies. Building upon the insights gained from these studies, the INE underwent revisions, leading to the development of the INE-R (Incivility in Nursing Education Revised) as documented in the work by Clark and colleagues in 2015.

The INE-R is structured into three parts:

Part I collects demographic information that can be customized according to the specific study’s requirements.

Part II encompasses a 48-item Likert-type questionnaire, divided into 24 items reflecting student behaviors and 24 items mirroring corresponding faculty behaviors. For instance, student behavior such as “Skipping class or other scheduled activities” corresponds to faculty behavior like “Canceling class or other scheduled activities without warning.” Participants are asked to rate the level of incivility on a 4-point Likert scale (1 = not uncivil, 2 = somewhat uncivil, 3 = moderately uncivil, 4 = highly uncivil) and to indicate how frequently they have observed each behavior during the past 12 months (or within a specified time frame) (1 = never, 2 = rarely, 3 = sometimes, 4 = often). The scale comprises two subscales: low-level incivility with 15 items (items 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 15, 16, 18) and high-level incivility with 9 items (items 13, 14, 17, 19, 20, 21, 22, 23, 24).

In Part III, there are four open-ended questions designed to elicit the perspectives and experiences of students and faculty members regarding academic incivility.

In the original scale’s validity study, the Cronbach’s alpha values for the total scale were ≥ 0.96 for student behaviors and ≥ 0.98 for faculty behaviors. The scale can be administered in a face-to-face format as well as in a web-based format [19].

Scores for INE-R items can be assessed individually, or they can be averaged to derive either a total score or distinct subscale scores, specifically for “lower level of incivility” and “higher level of incivility” items [19].

Translation process

The translation and adaptation of the scale adhered to the guidelines from WHO (2017) and Gregoire (2018) [24, 26, 27].

Step 1

The scale was translated by two independent translators fluent in Turkish and English, alongside five nursing faculty members. After reviewing the translations and original expressions, an initial Turkish version was created, which was then assessed by a Turkish Language and Literature specialist for alignment with Turkish language structure. Corrections were made based on the expert’s feedback (3 nurse academicians), followed by a language validity study. During the translation process, some terms required adjustment for cultural and linguistic appropriateness. Feedback from the Turkish Language and Literature specialist helped refine the translation to align with Turkish language structure. After necessary corrections, the final version of the scale was deemed linguistically valid for the study.

Step 2

The original scale, its translation, scoring method, and related documents were sent to 18 nursing experts to evaluate content validity using the Content Validity Index (CVI) as a criterion. Experts rated items on a scale from 1 to 4, with at least 80% of items needing to score between 3 and 4 for validity [28]. The scale received a CVI score of 91%, leading to revisions based on expert feedback.

Step 3

An independent native English speaker, proficient in the translated language and unaware of the scale, performed back-translation of the revised version.

Step 4

The back-translated version was synthesized. Then the final version was sent to Cynthia Clark for evaluation of meaning retention. Adjustments were made based on her recommendations, finalizing the Turkish draft.

Step 5

A cognitive interviewing was conducted with 10 nursing students from the 3rd and 4th years to evaluate question clarity. Purposive sampling was used. All participants found the items clear and the topic engaging. After the pilot study, the final Turkish version was created and sent to Dr. Cynthia Clark, with a serial number assigned at each stage to ensure clarity during data collection and publication [27].

Data analysis

The data obtained in the research were analyzed using the SPSS (Statistical Package for Social Sciences) for Windows 25.0 [29] and AMOS 21 [30] programs. Descriptive statistical methods (frequency, percentage, mean, standard deviation) were employed for data evaluation. ±3 z standardized values were calculated for data analysis, and normality was checked. Additionally, multivariate outliers were examined using Mahalanobis distance values. Outliers and extreme values were removed from the dataset. The statistical methods used for the validation and reliability of the INE-R are provided in Table 1.

Table 1 Statistical methods used in the research

Results

Demographics characteristics of the students

Data were collected from a sample of 522 nursing students. The students had a mean age of 21.32 ± 1.54 years. The majority of the students in the sample were female, constituting 87.7% of the participants. Additionally, 54.2% of the students were in their third year of study. Further details regarding the demographic characteristics of the students are provided in Table 2.

Table 2 Demographic characteristics of the students (n:522)

Validity

Confirmatory factor analysis (CFA) was conducted to ascertain the factor structure of the scale. The results of the structural equation modeling revealed the significance of the scale with 24 items and 2 factors at the p < 0.001 level, as depicted in Fig. 1. During model refinement, a covariance was introduced between errors with high MI (modification indices) values. The model’s fit indices were as follows: CFI: 0.97, RMSEA: 0.06, RMR: 0.03, SRMR: 0.04, IFI: 0.97, and chi-square/degree of freedom (χ²/df): 3.06, as outlined in Table 3.

In conclusion, the study determined that the scale’s construct validity was established, with factor loadings ranging from 0.47 to 0.74 for the lower level of incivility sub-dimension and from 0.91 to 1.00 for the higher level of incivility sub-dimension.

Fig. 1

The results of the first-level confirmatory factor analysis for the multi-factor model of INE-R

Table 3 The findings of the confirmatory factor analysis for INE-R

Reliability

The reliability analysis indicated that Cronbach’s alpha for the total INE-R Scale with its 24 items was 0.95. Furthermore, the lower level of incivility sub-dimension demonstrated a Cronbach’s alpha of 0.89, while the higher level of incivility sub-dimension exhibited a high reliability of 0.99.

In the item analysis, it was observed that the item-total score correlation values for all items exceeded 0.30, as presented in Table 4. These values ranged from 0.462 to 0.887. Additionally, it was noted that the removal of any item from the scale did not lead to an improvement in the Cronbach Alpha value.

Regarding the item analysis for the lower-level incivility sub-dimension of the scale, it was determined that all items exhibited item-sub-dimension correlation values exceeding 0.30, with values ranging from 0.489 to 0.662. These findings indicate that the items adequately represent the sub-dimension.

Similarly, for the higher level of incivility sub-dimension, the item analysis revealed that all items displayed item-sub-dimension correlation values above 0.30, varying between 0.910 and 0.992. Thus, it can be concluded that the items effectively represent the higher level of incivility sub-dimension.

Table 4 The results of the item analysis for INE-R

To assess the stability reliability, the scale was administered to 30 students at two-week intervals. The dependent samples t-test was utilized to compare the test-retest results, and no statistically significant difference was observed between the test-retest scores of the scale (p > 0.05). Specifically, the test-retest scores for the two sub-dimensions of the scale, lower level of incivility (t= -0.658, p = 0.516) and higher level of incivility (t= -0.787, p = 0.438), demonstrated that the scale reliably measures the same variable at different points in time.

The results so far have provided the outcomes of applying the 24-item faculty behavior form to the student population. The 24-item student behavior form, on the other hand, was administered to 240 nursing faculty members to conduct a construct validity analysis, and it was observed that the construct validity results for this form did not meet the desired level. Data of faculty members were collected from the same 11 universities on the same dates as the students. The INE-R Path Diagram and Factor Loadings are presented in Fig. 2. The factor loadings for the 15 questions forming the first factor (low-level incivility sub-dimension) range from 0.50 to 0.67, while for the 9 questions forming the second factor (high-level incivility sub-dimension), they vary between 0.20 and 0.98. According to the confirmatory factor analysis (CFA) conducted on the form containing student behaviors of the INE-R scale, it was observed that the goodness of fit values did not meet the acceptable fit criteria (Table 5). It was concluded that the structure of the form containing student behaviors in INE-R does not align with the original scale structure developed by Clark et al. (2015).

Fig. 2

INE-R path diagram and factor loadings

Table 5 INE-R confirmatory factor analysis results

Discussion

The original scale comprises two distinct forms, one focusing on student behaviors and the other on faculty behaviors. Depending on the respondent, the form provides the opportunity for student-student or faculty-student evaluations when assessing student behavior. Similarly, the form concentrating on faculty behaviors allows for student-faculty or faculty-faculty evaluations, contingent on who completes it. In this study, the form encompassing faculty behaviors was administered to students, resulting in student-faculty evaluations. For researchers aiming to utilize the scale, the Turkish-validated INE-R faculty form can be administered to students, facilitating student-faculty evaluations.

In validation and reliability studies of the INE-R scale conducted in various cultures, it has been observed that exploratory factor analysis (EFA) was carried out, resulting in the development of different models compared to the original scale [20,21,22]. However, in this study, the two-factor structure presented in the original scale through EFA has been tested in the Turkish cultural context with a new dataset using confirmatory factor analysis (CFA).

The language equivalence and validity, back-translation, pilot application, construct validity, and reliability analyses of the INE-R scale were conducted. It was found that the faculty form of the scale, the Turkish INE-R faculty form, was consistent with the original version by Clark et al. (2015), featuring 24 items and a two-dimensional structure. Fit statistics are interpreted using acceptable values to determine whether the model can be accepted [31].

A chi-square/degree of freedom (X²/Sd) value below 5 indicates acceptable model fit, while a value below 2 suggests excellent fit [32, 33]. In this study, it was found that the X²/Sd value of the scale was 3.06, indicating acceptable fit. In the original scale, this value was 2.90 for the faculty form [19], 3.23 for the Korean version [22], and 2.5 for the Iranian version [20]. It was observed that confirmatory factor analysis was not conducted in the study adapted to Arabic [21]. In the study, the CFI value of the scale was found to be 0.97. It is noted that a CFI value above 0.90 is acceptable, and a value above 0.95 indicates excellent fit [32, 33]. Therefore, it was found that the scale exhibited excellent fit in terms of the CFI fit index. In the original scale, this value was 0.93 [19], 0.93 in Abedini et al.‘s (2022) study, and 0.87 in De Gagne et al.’s (2016) study. In the current study, the RMSEA value was 0.06, indicating acceptable fit, and the SRMR value was 0.04, indicating excellent fit.

The reliability of the INE-R was assessed using Cronbach’s alpha reliability analysis, item-total score correlations, and test-retest analysis for stability over time. In our study, the Cronbach’s alpha values for the scale were calculated as follows: 0.89 for the low-level incivility subscale, 0.99 for the high-level incivility subscale, and 0.95 for the overall scale. In previous research, the Cronbach’s alpha values for the scale were reported as follows: 0.98 for the low-level incivility subscale in the original scale and 0.93 for the Korean version; 0.99 for the high-level incivility subscale in the original scale and 0.97 for the Korean version; and 0.99 for the overall scale in the original scale, 0.96 for the Korean version, and 0.87 for the Arabian version. Therefore, in this study, it was found that the Cronbach’s alpha values for the scale were consistent with the literature, indicating that it is a highly reliable scale within the range of 0.80-1.00 [32].

The item-total score correlation analysis indicated that all items in the scale were interrelated, measuring the intended qualities effectively. No items with a correlation value below 0.30 were identified, and therefore, no items were removed from the scale. It is noteworthy that the item-total score correlation analysis was not provided in the original scale and some other adaptation studies [19,20,21]. In the Korean adaptation study, it was observed that all items had item-total score correlation values above 0.30, with only one item, “Refusing or being reluctant to answer questions posed to them,” having a correlation value of 0.29, which was very close to the desired threshold. Consequently, no items were eliminated in the present study [22].

Limitations

This study is subject to several limitations. Firstly, as a consequence of the COVID-19 pandemic, data collection through Google Forms caused delays in obtaining institutional permissions, correspondence, and the data collection process, exceeding the initially planned timeline. Secondly, the study’s sample predominantly consisted of female students, which may affect the generalizability of the findings. Thirdly, although the validity and reliability results for the faculty behaviors form were satisfactory, the corresponding results for the student behaviors form did not meet the anticipated standards. Future research also can enlarge the sampling. Due to data collection strategy (%57,9 online), selection bias may have been generated. For example, the study may have attracted more attention of students with a high perception of incivility and participation of those may have increased accordingly. Study results are limited to the sample.

Conclusion

The results of this study demonstrate that the INE-R, with its 2 sub-dimensions and 24 items, is a valid and reliable tool for assessing the perceptions of Turkish nursing students regarding uncivil faculty behaviors and the frequency of their exposure to such behaviors. While there have been studies focusing on workplace incivility and bullying among nurses and student nurses, there is no other known study in Turkey addressing incivility in nursing education that encompasses all educational settings. This study is expected to draw attention to the concept of “civility” in Turkish nursing education and raise awareness towards establishing a culture of civility. By utilizing the INE-R in nursing schools, the current situation in terms of perceived incivility and its frequency can be assessed. As a next step, future research may consider developing protocols aimed at preventing incivility and fostering a culture of civility based on the acquired results. Furthermore, it is recommended that the structural validity of the INE-R student behavior form be re-examined with a new group of faculty members.