Receipt date: 31/12/2021 / Revision date: 27/02/2022 / Acceptance date: 29/05/2022

EL FORO VIRTUAL COMO IMPULSOR DE LA EXPERIENCIA DE APRENDIZAJE

Introduction

In the 21st century, humanity is facing the COVID 19 pandemic. According to Johns Hopkins University (2020), the impact on people's lives is not only due to the magnitude of the infected population, but also to the millions of deaths worldwide. This virus, which spreads mainly through human-to-human contact, restricted human activities to the maximum with extreme measures of physical distancing. 

In these circumstances, the transmission of knowledge at all levels of formal education has been affected, to a large extent, by the prohibition of face-to-face classes, which were replaced by remote, distance or online education. 

A new way of teaching and learning in a context of pedagogical, infrastructure and connectivity deficiencies implies a major challenge because it depends on multiple factors, such as teachers' professional training and Internet access. Regarding the effect of COVID-19 in Latin America, the International Labor Organization (2020) states:

Distance training or e-learning through the virtual platforms of institutions is proving to be an extremely useful tool, but it presents its own particular challenges in terms of teacher preparation and Internet accessibility for participants (p. 4).

Among these challenges, pedagogy within the virtual classroom is fundamental and a priority. Noriega and Torres (2011) argue that today's teachers, in addition to knowledge of their specialty, require training in didactics, pedagogy and, above all, in information technologies for virtual management.  This challenge is a great opportunity for interaction and interactivity of pedagogy in the virtual classroom. McLuhan's famous phrase, 'the medium is the message', is increasingly valid in view of the various forms of knowledge transmission, depending on the medium used (Roncallo-Dow, 2014).

In the case of master's degree students at the Graduate School of the Universidad Tecnológica del Perú (UTP), the quarantine decreed by the Peruvian government forced an abrupt change from face-to-face to remote classes, through a learning management platform integrated with a videoconferencing platform.

Pedagogy in the virtual classroom is different from the use of information and communication technologies (ICT). They complement the virtual context in the same way that the multimedia projector, blackboard and desks do in traditional face-to-face education. One of the difficulties, in face-to-face and distance classes, is to achieve the participation of all the students; that they express their opinion, debate ideas or ask for clarifications based on the teacher's exposition or their classmates' interventions. 

In face-to-face classes, students' verbal participation is limited by the time it would take to listen to everyone's opinions and at the same time fulfill the activities of the course syllabus. On the other hand, the passive participation of students may be a consequence of their fear of failure or simply due to personal shyness.

In this sense, remote education -both synchronous and asynchronous- has advantages over face-to-face education. If a face-to-face forum is understood as a meeting of people who discuss a certain subject, a virtual forum, according to Dockerty (2019), is the exchange of knowledge between people through the Internet. 

This study defines virtual forum as the meeting point of opinions of different people, who send and receive comments, located in different places and connected through the forum module of an LMS(Learning Management System ). Real-time interaction through an audiovisual collaborative work platform is defined as a chat or conversation of a more informal nature.

For Riggs (2020), it is an effective and efficient tool to obtain the opinion of all students and to promote teacher-student interaction, among the students themselves, and between them and the task. The forum as a component of a comprehensive inclusive learning strategy is cemented by the statement that "people often ask me about my favorite online teaching tool. My answer is always the same: without a doubt, it is the online discussion forums" (Darby, 2020, p. 2). 

However, it is necessary to know and systematically monitor whether the educational model is fulfilling its objective, whether there are gaps in the different sequences of instructional design, whether managers supervise the pedagogy in the classroom today more complex due to virtuality and ICTs that impact on the adaptation -both of teachers and students- to the new language of the environment (Rochera et al., 2021).

The innovation that the research provides is based on recommendations for teachers and instructional designers in remote education at the Graduate School of the Universidad Tecnológica del Perú to use, promote and enhance the forum, because the experience of social interaction impacts positively on the satisfaction of the students' learning experience.

It is also interesting to delve into the gaps and opportunities that limit the techno-pedagogical adaptation in online educational practice, such as the context that should be taken into account for the instructional design of the forum, feasible design, development and implementation alternatives for active and collaborative learning, and which learning and assessment objectives can be defined to impact the learning experience.

Unlearning to learn is a great opportunity for interaction and interactivity of pedagogy in the virtual classroom. In this regard, Aparisi (2020) explains: 

We found that interaction will usually be a symmetrical process, between people and strictly social. Interactivity, on the other hand, will be an asymmetric process, where a dialogue is proposed between certain artifacts, whether books, software or activities, and the students (p. 15).

In order to understand the phenomenon of interactivity, Zangara and Sanz (2012) express that there is interactivity in the structure (course design and materials), and there is interaction between people in the dialogue through the mediating activity of technology.

The role of the teacher in virtual education involves acquiring new concepts, applying new procedures and accepting new ways of acting reflected in habits, customs and preconceived ideas in the classroom. Boumadan et al. (2020) state that teachers must have good techno-pedagogical training, which allows them to have control in the implementation and innovative experience. For teachers, unlearning to learn is an opportunity to assume that things can be done differently and obtain the same or better results, as pointed out by Rangel and Peñalosa (2013): "a new type of literacy is required, which some authors have agreed to call digital literacy" (p. 11). Another opportunity for the new teacher is to manage learning by putting the learner at the center (González et al et al., 2014). It is true that this is a process that involves the efforts of a committed and student-oriented facilitator. As indicated by Llorente (2006), teachers must be able to connect with their students by establishing relationships, clarifying doubts, and encouraging participation. 

On the other hand, teaching should be student-centered (Xiaowei and Zhang, 2020; Gonzales et al., 2014). Likewise, Darby (2020) mentions six ways to encourage online forums: 1) participate in the discussion, 2) clarify doubts, 3) praise good interventions, 4) clarify misconceptions, 5) guide the discussion toward the learning objective, and 6) strategically manage quantity and timing so that students know the teacher is with them. 

About online forum-based (synchronous) learning, Xiaowei and Zhang (2020) argue that it complements traditional learning as it can be divided into sessions that help students proactively complete assignments and avoid plagiarism. In addition, it improves student engagement and motivation, as most believed that their presentation and academic skills could be improved through communication and practice in the online forums. 

Wilkins (2002) reveals some advantages of using discussion forums: flexibility of readings and posting comments at any time and place in the world. The asynchronous modality allows for deeper and more critical reflections in the written dialogue. It provides a more relevant experience for introverted students, as they feel more confident doing it online. Despite this, he notes a disadvantage: participants expect an immediate response to their interactions and this can lead to some distress and frustration.

The forum allows for reflection, research and the development of critical thinking by sharing opinions. Ornellas (2007) refers to "reinforcement of learning, knowledge of attitudes towards certain topics, development of social skills and written communication" (p. 2). According to Brown (2015), the forum is a good starting point for feedback, which impacts student performance. The author proposes to acknowledge the student's contribution with a quick response adding content, perspectives or experience and, above all, challenging to continue the conversation. For his part, Birch (2015) offers two techniques: the first, related to instilling social purpose, i.e., the forum is for students to share their experiences and experiences. The second technique uses stimuli to reward punctuality, quality, courtesy and absence of errors. 

It must be remembered that each person has his or her own DNA; no two brains are alike and, therefore, no two people express themselves, motivate themselves and learn in the same way. If individuals require different treatment, different students also require educational treatment according to their needs. With these premises in mind, the Universal Design for Learning (UDL) approach, developed by the Center for Applied Special Technology, makes it possible to understand and manage these differences in instructional design. About the SAD, Alba Pastor et al. (2011, p. 21), express that "there is no one optimal representation medium for all students; therefore, it is critical to provide multiple options when delivering information."

Belloch (2012) states:

The ADDIE model is an interactive instructional design (sic) process, where the results of the formative evaluation of each phase can lead the instructional designer (sic) back to any of the previous phases. ADDIE is the basic model of instructional design (sic), as it contains the essential phases of instructional design: analysis, design, development, implementation and evaluation (p. 10).

Reigeluth (2016) highlights two challenges for the roles of teacher and student. On the one hand, that of the creator, facilitator and mentor of the learning experience by the teacher and, on the other hand, that of the hardworking, responsible, autonomous, co-instructor and co-creator student. For their part, Góngora Parra and Martínez Leyet (2012) mention that "learning design models based on constructivist theories are more suitable for the new educational contexts and offer more opportunities to design training actions that enable the achievement of professional competencies" (p. 356). For Sharif and Cho (2015), the instructional designer is an architect because he/she designs, engineer (teacher) because he/she builds and manages the project by integrating different knowledge into learning strategies or pathways using new information and communication technologies. Its distinguishing features should be flexibility, creativity, collaboration, innovation and adaptation. 

Seethamraju (2014) and Castro Méndez et al. (2016) highlight virtual forum peer work, and describe an interesting experience on the benefits of discussion forums and peer-to-peer learning to enhance learning, both in a traditional face-to-face and online environment. They conclude that the combined use of discussion forums with the traditional method improves the quality of cognitive learning. Despite this, they highlight the significant workload involved in the creation, monitoring and evaluation of online discussions by faculty, so a design and structure with appropriate faculty workload policies needs to be put in place. In this sense, what Rochera, et al. (2021) comment on feedback on student actions is key to the satisfaction levels of a forum discussion. 

Harris and Sandor (2007) state that forums motivate students in online learning by encouraging them to take a more active and central role. Wang (2017) argues that designing engaging and progressive problem-solving activities is at the heart of the effective classroom method. In addition, self-reflection and self-assessment activities in the flipped classroom attract participation and social interaction, which, in turn, promotes participation in problem-solving activities. In his research on interactive learning in virtual forums in a postgraduate context, Morales (2017) confirms the necessary role of the teacher in the design and proactivity, as well as the communication style in which the courses are taught, preferably blended courses.

Reichheld (2003) developed the Net Promoter Score (NPS) method to measure customer satisfaction with a company's goods or services. It is also an index that correlates the reputation, growth and sustainability of a company's business model over time, as it measures the trend to verify whether the organization is moving in the right direction. 

Despite Kristensen and Eskildsen's (2014) criticisms of the NPS model, their position is in contrast to the various studies by Lee (2018), Fisher and Kordupleski (2019), Nakwong et al. (2020) and Srirahayu et al. (2021), which use NPS to measure satisfaction. Rego et al. (2013) clarify that market share is a strong negative predictor of future customer satisfaction. In turn, Otto et al. (2020) argue that satisfaction is the marketing strategy that best describes firm performance outcomes. What is concrete is that the NPS experience satisfaction measure has not been surpassed or challenged to this day. For example, the U.S. company Qualtrics is an expert in measuring customer experiences such as NPS. Qualtrics services are used by institutions, such as Stanford University - one of the world's leading teaching and research universities - which uses Qualtrics as a standard because of its practicality, convenience and because it allows them to be more productive (https://www.qualtrics.com). 

NICE Systems, Inc, co-developer of NPS, systematically publishes a comparison of leading brands by industry. The research behind the Net Promoter methodology shows that companies with scores higher than their competitive set grow faster and are more successful (https://www.nice.com/). Customer Guru, a European Community company, provides automated customer satisfaction measurement services using the Net Promoter System. According to this company, the average for the education sector is 54. The top universities for them are Wharton School of Business, Harvard Business School and Duke's Fuqua School of Business, each with 51, 41 and 67 NPS points, respectively (https://customer.guru/).

The research has academic-practical relevance because of the results that can be shared with instructional designers, academic managers, teachers and researchers interested in adapting to the new experience of remote education that contribute to the social interaction and satisfaction of the learning experience between teacher and students (Buil et al., 2012).

Method

The study population included male and female master's degree students at the Graduate School of the Universidad Tecnológica del Perú, between 2019 and 2021, over 25 years of age and residents of Peru. The methodology was mixed qualitative and quantitative

Instruments

Instrument 1: Reichheld's NPS (2003). Metric or performance indicator useful for monitoring over time the ability to satisfy and retain students. In practice, it is used to understand how satisfied they are and whether they are promoters or detractors. According to Baehre et al. (2021), with this methodology, the forum participants are offered response options from 0 to 10. A detractor is defined as an individual whose rating ranges from 0 to 6 and who was not satisfied with the learning experience. On the other hand, the promoter is the one who rates between 9 and 10, which indicates a high level of satisfaction that even drives and promotes the experience. Neutrals are those who give a rating of 7 and 8 (they are moderately satisfied with the experience). The NPS score is the difference between the score obtained by promoters and detractors. Neutrals are not taken into account in the calculation. NPS score ranges can vary from -100 (all are detractors) to +100 (all are promoters). A rating above 0 is perceived as positive. 

The objective of this measurement is to evaluate the perception of satisfaction with the educational experience of participating in the forum and the recommendation of the forum to their colleagues. The survey was anonymous and voluntary, and was conducted virtually synchronously at the end of the last class of each course. Three closed questions were planned: 1) how would you rate your experience in the virtual classroom forum of this course, 2) how likely are you to recommend to your colleagues and friends to go through the same virtual classroom forum experience of this course, and 3) how likely are you to recommend to your colleagues and friends to take this course as it has been combined? Additionally, open-ended questions were used to identify the reasons for each assessment. 

Instrument 2: Academic survey addressed to students through a standard questionnaire approved, used and carried out by the Academic Direction of the Graduate School. The objective is to evaluate the course and the teacher in terms of quality and satisfaction. The survey was anonymous and voluntary, and was conducted virtually synchronously on the last day of class of each course. The sample consisted of 491 respondents from the target population who met the formal condition of being registered in certain courses and types of master's degree (see Table 1).

Table 1

Master's degrees, courses and respondents

#	Course	Master's Degree	Quantity	Surveys	%
1	Loyalty and Engagement	Marketing and Commercial Management	6	115	23,42
2	Strategic Direction	Operations and Logistics Management	4	80	16,29
		Project Management	4	80	16,29
3	Competitive Strategies for Changing Times	Project Management	4	65	13,24
4	Strategic Commercial Management	Operations and Logistics Management	3	34	6,92
5	Go Strategy	Business Administration (MBA)	3	57	11,61
		Customer Intelligence	2	30	6,11
6	Customer Relationship Management	Customer Intelligence	2	30	6,11
			28	491	100,00

A total of 28 courses were given, accumulating 560 class hours divided into four-hour weekly sessions (82.28%) and 16-hour weekend sessions on Saturdays and/or Sundays (17.72%). Finally, for the analysis of quantitative and qualitative results, the Excel pivot table and MAXQDA were used, respectively.

Results

The first instrument on the experience and recommendation of the forum showed favorable and positive results; in all three cases the NPS exceeded 50 points, obtaining a weighted average of the three questions of 51.66 (see Table 2).

Table 2

Questions and NPS score

	Questions	NPS	Promoters	Neutrals	Detractors
1	how do you evaluate your experience in the virtual classroom forum of this course?	51,22	58,59	34,04	7,37
2	how likely are you to recommend the same virtual classroom forum experience of this course to your colleagues and friends?	51,22	61,44	28,34	10,22
3	how likely are you to recommend to your colleagues and friends to take this course as it has been combined?	52,44	61,67	29,2	9,13
	Total, weighted	51,66	60,57	30,53	8,90

Note. Own elaboration based on 491 respondents per question

 

Exploratory analysis of promoter words. After selecting the most frequent words and their combinations, two groups of words and the concepts related to them were identified. The first relates to interaction, sharing ideas and opinions, getting to know each other. The second is linked to the good experience of the virtual forum tool and the methodology used. 

When analyzing the reason for their NPS assessment (also called management with promoters), they recognize the good experience and the opportunity to share opinions with peers as an input for meaningful learning. In the open-ended responses, two main reasons were identified: exchange of ideas and methodology, which together account for 54.4%. A second group of reasons is the course, learning, experience and teacher experience, with 37% together. 

Exploratory word analysis of neutrals. The analysis identifies a first group that accepts with good aptitude the virtuality due to the circumstances of the pandemic. The second group proposes improvements to the forum, such as increasing interaction, sharing opinions and different points of view with other colleagues. The reason for your NPS rating (also called management with neutrals) consolidates opinions about the good experience and the opportunity to share opinions with peers as input for better learning. In the open-ended responses of neutrals, the first rating is the teaching method (20.6%), contributions of opportunities for improvement (17.4%), recognition of learning achieved (12.9%), interest in greater exchange of ideas in the forum (11.8%), and those who prefer face-to-face classes (10%). Together, these five reasons account for 72.7% of the neutral ratings. 

Exploratory analysis of detractors' words. Two groups were identified. The first, related to the acceptance of virtuality, but prefer face-to-face. The second has to do with encouraging interaction, discussion and participation of colleagues in the forum. When analyzing the reason for their NPS rating (also called management with detractors) they highlight the good experience and the opportunity to share opinions with peers for better learning. In the open-ended responses, only 33% of the detractors (8.9% of the total) preferred the face-to-face modality to the remote or online modality. 

Analysis of Instrument 1 - NPS Survey by Age Group.  The weighted NPS index of instrument 1 was very positive (50.1%); however, there were significant differences between age groups: the older the age group, the rating progressively declined; the younger the age group, the higher the rating. The two youngest age groups, between 25 and 35 years of age, exceed the overall average out of 100 (see Table 3).

Table 3

NPS by age group

Age group	NPS	Promoters	Neutrals	Detractors	%
46 a +	45,66	53,76	38,15	8,09	91,14
36 to 45 years old	48,22	58,70	30,83	10,47	96,25
26 to 35 years old	51,75	58,75	34,24	7,00	103,29
Up to 25 years	60,47	69,77	20,93	9,30	120,70
Total	50,10	58,47	33,15	8,37	100,00

Cross-sectional analysis of instrument 2 - UTP Academic Survey. The results refer to the following four areas: teacher performance, methodology by activities, methodology by resources and overall assessment. The vigesimal score indicates the valuation in each of the variables evaluated by the academic area of the School. The teacher performance factor obtained an approval rating of 18.05, the methodology by activities, 17.81, the methodology by educational resources 17.97 and, finally, the general evaluation obtained 17.98. In sum, an overall favorable and positive assessment. The results by course show that there are no significant differences (see Table 4).

 

Table 4

Academic Evaluation Results - UTP Course Summary by Course

UTP Academic Survey				Weighted evaluation
Courses		#	%	Teacher performance	Course methodology Activities	Course Methodology Educational Resources	Overall appraisal
1	Strategic Direction	145	29,53	18,44	18,42	18,40	18,56
2	Competitive Strategies in Times of Change	80	16,29	17,58	17,20	17,38	17,35
3	Strategic Commercial Management	34	6,92	18,10	17,54	18,37	17,74
4	Loyalty and Engagement	115	23,42	17,76	17,91	18,21	18,00
5	Customer Relationship Management	30	6,11	18,50	17,90	18,08	18,45
6	Go Strategy	87	17,72	17,68	17,26	17,17	17,43
Totals		491	100,00	17,99	17,80	17,95	17,97

Note. Surveys of the Graduate School of the Universidad Tecnológica del Perú, 2019-2021.

Opportunities for improvement. Opportunities for improvement refer to: content (31.3%), methodology (35.4%), interaction (10.4%), more hours (14.6%) and better attendance (8.3%). Together, these five reasons account for 100% of the ratings of the opportunities for improvement. 

Qualitative analysis of instruments 1 and 2. The percentage distribution of instrument 1 versus instrument 2 is 85.2% and 14.8%, respectively. In the first instrument, 60.8% of the comments were from promoters, 8.7% from detractors and 30.5% neutral. The composition in the second instrument is 80.3% for positive comments and 19.7% for opportunities for improvement (see Table 5).

Table 5

Comment analysis - Instruments 1 and 2

Type	Number of comments	% vs subtotal 1	% vs. total	Instrument
NPS Survey
Promoters	855	60,80	51,80	1
Detractors	122	8,70	7,40	1
Neutrals	429	30,50	26,00	1
Subtotal 1	1406	100,00	85,20
UTP Survey
Positives	196	80,30	11,90	2
Opportunities for improvement	48	19,70	2,90	2
Subtotal 2	244	100,00	14,80
Total	1650		100,00	1 y 2

Discussion and conclusions

It is possible to affirm that the virtual forum is a positive driver in the satisfaction of the learning experience, because the social interaction that allows sharing ideas and opinions among peers, getting to know each other, receiving feedback from the teacher and consolidating learning is valued favorably and positively. However, the forum is part of the total experience provided by the teacher in terms of the dynamics of the methodology, the theoretical-practical knowledge and the virtual pedagogical experience of the teacher, all of which were highly valued by the respondents. More and better participation of other colleagues in the forum is recognized as an opportunity for improvement. 

These results are in agreement with those reported by Wilkins (2002), Harris et al. (2007), Zangara and Sanz (2012), Brown (2015), Riggs (2020), Aparisi (2020) and Darby (2020) on interaction as a strictly social symmetric process between people. The virtual forum, whether synchronous or asynchronous, is an eloquent demonstration of this statement, since it is flexible in time and place, allows for deeper reflections and responses with feedback, and promotes collaborative learning. In contrast, the interactivity they also propose is the communication between the user and the digital product, not always created with good training practices.

It is also worth noting what Llorente (2006), Góngora Parra and Martínez Leyet (2012), Rangel and Peñalosa (2013), Seethamraju (2014), Reigeluth (2016), and Morado (2017) state that it takes time to accompany teachers in their digital pedagogical transformation, since it takes time for various reasons, from resistance to change to typical fears of unlearning in order to learn. This coincides with the students' positive assessment of the updating of the teacher's competencies.

Alba Pastor et al. (2011) explains that there is no one optimal representation medium for all learners; it is critical to provide multiple options when delivering information. Agree with the need to design a forum with disruptive creative alternatives, but that meets the educational objective as pointed out by Sharif and Cho (2015) and Morales (2017).

However, the research findings show partial agreement with the two techniques of Birch (2015). The first consists of instilling social purpose as a tool for all students to make it their own in solidarity. The second is to provide incentives to reward participation in terms of punctuality, quality or absence of errors. 

The position described by Birch (2015) in his first technique responds to a constructivist culture on pedagogy in the classroom, based on stimulating the group's solidarity awareness, fostering the spirit of competition or detecting who should try harder. The second technique not only recognizes that not all students respond to these stimuli because they don't care, don't want to be there, are there out of obligation, or show disinterest in learning. His proposal is clearly a behaviorist one of positive and negative reinforcement. In the Peruvian context, the little participatory, uncompetitive and passive attitude of not very few students is the usual and regular behavior, for which it is necessary to take into account the two currents of thought in accordance with the concept of universal design for learning (UDL).

The findings allow us to affirm that the resulting NPS index is favorable and positive, because it reached 51.66 points above universities of global prestige. The main reason was that the forum allowed the exchange of ideas for better learning. Even for the detractors, the forum was a good tool, although they expected their peers to participate more actively and frequently.

Second, the NPS scores by age group show that the higher the age, the slightly lower the rating. However, this does not imply total dissatisfaction with remote education. In practice, the older age group finds it difficult to adapt to collaborative virtual dynamics; proof of this is the technical support of their own younger peers in the middle of a synchronous class.

Finally, the positive results of the four dimensions evaluated by the same educational institution support that the use of the forum influences the satisfaction of the learning experience. 

With regard to the gaps and opportunities in the use of the forum, the following aspects stand out as relevant:

Lack of understanding of the objectives and importance of the forum. A constant reminder is needed about the impact on the final evaluation, either by not participating, by doing it in a mediocre way or by doing it for the sake of compliance.  

Resistance to the virtual modality. Although resistance only reached 2.76 points of the total surveyed, it must be accepted that this is a time of generational transition in which it is necessary to support inclusion and reinforce support for students in their virtual adaptation. 

Passive participation. Without pretending to generalize, it responds more to personal and Peruvian idiosyncrasies, a situation that is beyond the teacher's responsibility without this meaning a decline in his or her motivational mission.  

Interaction. The most valued attribute in the use of the forum is to be able to interact and get to know different points of view from other students because it reinforces knowledge, compares levels of reflection, saves time and encourages debate for better learning based on one's work experience. The frequent practice of written dialogue becomes a habit and, therefore, a natural habit. 

Pedagogical complement. The possibility of comprehensive learning, attitudinal as well as conceptual and procedural, is enhanced if design and execution are aligned. 

Synergy of styles. Maintaining the combination of constructivist and behaviorist models depends on each group, although teaching practice requires a dose of positive reinforcement and negative reinforcement typical of the behaviorist model.  

The didactic strategy. The themes of the forums should be triggers for greater participation. The forum is an excellent pedagogical tool also for the face-to-face modality. In this sense, the subject matter should be creative and diverse, and should minimize the traditional task of copying and pasting the correct answer. In short, it should encourage reflection and critical thinking through memorable impressions that stimulate comparison based on audiovisual stimuli, such as film sequences, the case method, the generation of controversies based on opposing positions, the application of models of analysis of the environment or correlating the conceptual with the application of a personal work practice.

About the context for the instructional design of the forum. Identifying what context, the instructional design of the virtual forum should take into account is essential. From the instruments, such as the master's profile, the specific course, the student directory, the activity prior to the start of the course, the interventions in the first forums and the interactions in class it is possible to identify the context for the initial design, but at the same time adjust details dynamically during the course, even if it is a generic one for several master's degrees.

Limitations

The NPS of the education sector refers to the performance of the university educational center in general and not to an NPS index exclusive to the use of the forum. Despite this, this limitation does not invalidate the study because the rated index alone has a range internationally accepted as good. 

The NPS scale may encounter limitations in relation to the rigorousness or generosity of the valuation that different human groups in different regions of the world may have about different concepts. 

Continuity proposals

To assess the adaptive capacity of different age groups to the use of ICTs and the impact on their meaningful learning, with particular attention to adults over 50. 

To evaluate whether there is any correlation of the results of the international PISA tests conducted by the Organization for Economic Cooperation and Development (OECD). Peru ranked 64th out of 77 countries, with a passive attitude and little capacity for processing and critical reflection.

References

Alba Pastor, C., Sánchez Serano, J., & Zubillaga del Río, A. (2011). Diseño Universal para el Aprendizaje (DUA). Pautas para su introducción en el currículo. http://www.educadua.es/doc/dua/dua_pautas_intro_cv.pdf. 

Aparisi, L. (2020). Modelos pedagógicos en la educación virtual. Universidad Nacional Arturo Jauretche. https://doi.org/10.13140/RG.2.2.20208.81924 

Baehre, S., O’Dwyer, M. O’Malley & Lee, N. (2021). The use of Net Promoter Score (NPS) to predict sales growth: insights from an empirical investigation. Journal of the Academy of Marketing Science, 50, 67–84. https://doi.org/10.1007/s11747-021-00790-2.

Belloch, C. (2012). Diseño Instruccional. Unidad de Tecnológica Educativa de la Universidad de Valencia. https://www.uv.es/bellochc/pedagogia/EVA4.pdf.  

Birch, T. (2015). Techniques to Encourage Continuously Interactive Online Discussions. https://elearningindustry.com/encourage-continuously-interactive-online-discussions 

Boumadan, M., Soto-Varela, R., Ortiz-Padilla, M., & Poyatos-Dorado, C. (2020). What factors determine the value of an online teacher education experience from a teacher’s perspective? Sustainability, 12(19), 8064. https://doi.org/10.3390/su12198064 

Brown, J. (2015). The ABCs Οf High Quality Online Discussions. How to improve online discussions. https://elearningindustry.com/abcs-high-quality-online-discussions.    

Buil, I., Hernández, B., Sesé, F., & Urquizu, Pilar (2012). Los foros de discusión y sus beneficios en la docencia virtual: recomendaciones para un uso eficiente. INNOVAR. Revista de Ciencias Administrativas y Sociales, 22(43), 131-143. https://www.redalyc.org/articulo.oa?id=81824123012 

Castro Méndez, N., Suárez Cretton, X., & Soto Espinoza, V. (2016). El uso del foro virtual para desarrollar el aprendizaje autorregulado de los estudiantes universitarios. Innovación Educativa, 16(70), 23-41. http://www.scielo.org.mx/scielo.php?script=sci_arttext&pid=S1665-26732016000100023&lng=es&tlng=es. 

Darby, F. (2020). The Secret Weapon of Good Online Teaching: Discussion Forums. The chronicle of higher education. https://www.chronicle.com/article/the-secret-weapon-of-good-online-teaching-discussion-forums 

Dockerty, K. (2019). Developing pre-service teacher knowledge using online forums: supporting confident and competent teaching practices. Journal of Education, 45(4), 1-15. https://doi.org/10.1080/02607476.2019.1639260.  

Fisher, N. & Kordupleski, R. (2019). Good and bad market research: A critical review of Net Promoter Score. Applied Stochastic Models in Business and Industry, 35(1), 138-151. https://doi.org/10.1002/asmb.2417 

Góngora Parra, Y. & Martínez Leyet, O. L. (2012). Del diseño instruccional al diseño de aprendizaje con aplicación de las tecnologías. Revista Teoría de la Educación: Educación y Cultura en la Sociedad de la Información, 13(3), 342-360. https://www.redalyc.org/pdf/2010/201024652016.pdf. 

González, J., Cantu, M., & Maldonado, J. (2014). The virtual forums as teaching strategy in the context of Integral Upper Secondary Education Reform in Mexico. In Decima Tercera Conferencia Iberoamericana en Sistemas, Cibernética e Informática, Undécimo Simposium Iberoamericano en Educación, Cibernética e Informática, SIECI.

Harris, N. & Sandor, M. (2007). Developing online discussion forums as student centred peer e-learning environments.

https://www.researchgate.net/publication/29466076_Developing_online_discussion_forums_as_student_centred_peer_e-learning_environments  

Johns Hopkins University. (2020). Covid 19-Map. Coronavirus Resource Center. https://coronavirus.jhu.edu/map.html. 

Kristensen, K. & Eskildsen, J. (2014). Is the NPS a trustworthy performance measure? The TQM Journal, 26(2), 202-214. https://doi.org/10.1108/tqm-03-2011-0021 

Lee, S. (2018). Net promoter score: Using NPS to measure IT customer support satisfaction. Association for Computing Machinery. https://doi.org/10.1145/3235715.3235752 

Llorente, M. (2006). El tutor en E-learning: aspectos a tener en cuenta. Edutec. Revista 

Electrónica de Tecnología Educativa, 20, a060. https://www.edutec.es/revista/index.php/edutec-e/article/view/517 

Morado, M. (2017). El acompañamiento tecnopedagógico como alternativa para la apropiación de tecnología en docentes universitarios. Revista Actualidades Investigativas en Educación 17(3), 1-24. https://www.scielo.sa.cr/pdf/aie/v17n3/1409-4703-aie-17-03-00190.pdf 

Morales, A. (2017). Aprendizaje interactivo. Foros virtuales de aprendizaje en un contexto de postgrado [Tesis doctoral], Universidad de Alcalá. https://dialnet.unirioja.es/servlet/tesis?codigo=158745 

Nakwong, P., Muttamara, A., & Soontorn, B. (2020). NPS better predict online classroom.

Noriega, L. & Torres, E. (2011). Aulas Virtuales: ¿Desarrollo pedagógico y didáctico o avance tecnológico? [Trabajo de Grado], Universidad Militar Nueva Granada. https://repository.unimilitar.edu.co/bitstream/handle/10654/5383/NoriegaFrontadoLuisAntonio2011.pdf?sequence=2&isAllowed=y- 

Ornellas, D. (2007). El uso del foro de discusión virtual en la enseñanza. Universidad Autónoma de Guadalajara.  https://doi.org/10.35362/rie4442226 

Organización Internacional del Trabajo. (2020). El rol de la formación profesional    frente a los efectos del COVID 19 en América Latina. https://www.oitcinterfor.org/publicaciones/oitcinterfornotas/Fp_covid 

Otto, A., Szymanski, D., & Varadarajan, R. (2020). Customer satisfaction and firm performance: insights from over a quarter century of empirical research. Journal of the Academy of Marketing Science, 48, 543-564. https://doi.org/10.1007/s11747-019-00657-7  

Rangel, A. & Peñalosa, A. (2013). Alfabetización digital en docentes de educación superior: construcción y prueba empírica de un instrumento de evaluación. Píxel- Bit. Revista de Medios y Educación, 43. https://dialnet.unirioja.es/servlet/articulo?codigo=5769593  

Rego, L., Morgan, N., & Fornell, C. (2013). Reexamining the market share-customer satisfaction relationship. Journal of Marketing, 77(5), 1-20. https://doi.org/10.1509/jm.09.0363 

Reichheld, F. (2003). The one number you need to grow. Harvard Business Review. https://hbr.org/2003/12/the-one-number-you-need-to-grow 

Reigeluth, C. (2016). Instructional Theory and Technology for the New Paradigm of Education. Revista de Educación a Distancia, 50(1a). http://www.um.es/ead/red/50/reigeluth_esp.pdf 

Riggs, S. (2020). Student-Centered Remote Teaching: Lessons Learned from Online Education. EDUCAUSE Review. https://er.educause.edu/blogs/2020/4/student-centered-remote-teaching-lessons-learned-from-online-education 

Rochera, M., Engel, A., & Coll, C. (2021). The effects of teacher’ feedback: a case study of an online discussion forum in Higher Education. RED. Revista Educación a Distancia, 21(67). http://dx.doi.org/10.6018/red.476901 

Roncallo-Dow, S. (2014). Marshall McLuhan. El medio (aún) es el mensaje 50 años después de comprender los medios. Revista Palabra Clave, 17(3), 582-588.

https://www.redalyc.org/pdf/649/64931834017.pdf 

Seethamraju, R. (2014). Effectiveness of Using Online Discussion Forum for Case Study Analysis. Education Research International, 2004. http://dx.doi.org/10.1155/2014/589860 

Sharif, A. & Cho, Sunah (2015). Diseñadores instruccionales del siglo XXI: cruzando las brechas perceptuales entre la identidad, práctica, impacto y desarrollo profesional. RUSC. Universities and Knowledge Society Journal, 12(3), 72-86. https://www.redalyc.org/pdf/780/78038521006.pdf 

Srirahayu, D., Anugrah, E. y Layyinah, K. (2021). Influence of satisfaction and loyalty on Net Promoter Score (NPS) in academic libraries in Indonesia. Library Management, 42(6/7), 325-339.  https://doi.org/10.1108/LM-06-2020-0090ISSN01435124 

Wang, F. (2017). An exploration of online behaviour engagement and achievement in flipped classroom supported by learning management system. Computers and Education, 114, 79-91. https://doi.org/10.1016/j.compedu.2017.06.012 

Wilkins, B. (2002). Facilitating Online Learning: Training ta’s to Facilitate Comm [Proyecto de maestría, Brigham Young University]. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.95.1876&rep=rep1&type=pdf 

Xiaowei, K. y Zhang, W. (2020) An experimental case study on forum-based online teaching to improve student’s engagement and motivation in higher education. Interactive Learning Environments. https://doi.org/10.1080/10494820.2020.1817758 

Zangara A. & Sanz, C. (2012). Aproximaciones al concepto de interactividad educativa. JAUTI 2012. In I Jornada de Difusión y Capacitación de Aplicaciones y Usabilidad de la Televisión Digital Interactiva. Universidad Nacional de La Plata. http://sedici.unlp.edu.ar/bitstream/handle/10915/25943/Documento_completo.pdf?sequence=1