MLS PROJECT, DESIGN AND MANAGEMENTwww.mlsjournals.com/
How to cite this article:
Miró Vera, Y. A. (2019). Diagnostic of Self-Assessment of Students of the Design Module from the Projects Area of FUNIBER about IPMA 4.0 Competences Certification. Project, Design and Management, 1(1), 49-64. doi: 10.35992/mlspdm.v1i1.161
Abstract. This contribution is presented into the framework of the project for Updating the Design Module of FUNIBER based on the IPMA 4.0 certification. The aim of this proposal is to describe the diagnostic of students at the starting point of the course, based on a sample of 36 students from the total population. The methodology employed is of non-experimental design, exploratory with mixed perspective, and involved the application of a survey of 29 questions. Each question was related to one of the 28 IPMA competences in the main 3 areas (Perspective, Persons and Practice), except by Team work, which was related to 2 questions. The results have been analysed using Main Components Analysis and scattering plots. The main conclusion from statistical analysis is that the studied sample of students follows a trend with maximum frequency of answers in the level Medium, in a 92% of the competences considered. For the competence of Personal Integrity and Confidence a distribution with maximum in the level High was found, and for Results Orientation an equal distribution between Low and High levels was described. Finally, an optimization of the measurement instrument is proposed, based on statistical tools and the perspective described by Tobón in 2008.
Keywords: Diagnostic, project management, self-assessment, competences, IPMA 4.0
Palabras clave: Diagnóstico, gestión de proyectos, autoevaluación, competencias, IPMA 4.0
This work is part of the project to update the Design Module of the Ibero-American University Foundation from now on, FUNIBER, based on the International Project Management Association certification, IPMA 4.0. The final objective of this study is the proposal of a new version of the teaching materials used in the Design Module for the Creation of Projects provided by FUNIBER, that contemplates in an integral way the competences described by IPMA 4.0. In this first part of the work, the initial diagnosis of the students who have not yet completed the module is presented. The goal of this section is to establish a baseline reference point for the subsequent comparison of the performance of this same sample of students once the module in question has been completed, using the measurement instrument in order to identify the required modifications in the contents of the teaching material. This proposal is based on the action-research methodology, focused on education, which “is based on the fact that theory is developed through practice and modified through new actions”. (Latorre, 2003, p.10).
The main objective of this research was to produce a self-evaluation diagnosis of the level of competences of student from the Design Module in the Projects area of FUNIBER 2018, taking as reference the competences of IPMA 4.0, in order to meet part of the objectives of the broader research mentioned above.
Definition of Competence
According to the Real Academia Española competence means “expertise, aptitude or suitability to do something or intervene in a particular matter”.
The first use of the concept is in Plato's work (Lysis 215 A, 380 DC). The root of the word is "ikano", a derivative of "iknoumai", which means to arrive. The ancient Greek had an equivalent for competition, which is ikanótis (ικανοτησ). It is translated as the quality of being ikanos (capable), having the ability to achieve something, dexterity Epangelmatikes ikanotita means capacity or professional/vocational competence (Mulder, Weigel, & Collings, 2008, p.2).
They also mention:
Even earlier in time, the Code of Hammurabi (1792-1750 BC) mentions a comparable concept. In the Epilogue, a text translated into French can be read: “Telles sont les décisions e justice que Hammurabi, le roi compétent, a établies pour engager le pays conformément à la vérité et à l’ordre équitable.” Competition even appeared in Latin in the form of competens which was conceived as being able and allowed by law/regulation, and in the form of competentia, understood as capacity and permission. In the 16th century the concept was already recognized in English, French and Dutch; at the same time the use of the words competence and competency is dated in Western Europe. So, it is clear that the concept of competence has a long history, so it is not surprising that being professionally competent, sufficiently capable and able to perform certain tasks has been an aspiration throughout the ages. (Mulder, Weigel, & Collings, 2008, p. 4).
We can see that the concept of competence has a long history and that in order to achieve the great works of humanity it has been required that people be able to perform or perform certain tasks, even if the concept was not defined within a framework such as that currently envisaged.
Some definitions of competition highlighted by Mulder et al. (2008) and considered appropriate for this research are the following: “aptitude to face effectively a family of analogous situations, mobilizing consciously and in a fast, pertinent and creative way, multiple cognitive resources: knowledge, capacities, micro-competences, information, values, attitudes, schemes of perception of evaluation and reasoning" (Perrenoud, 2004 quoted by Mulder et al., 2008). "A set of knowledge, procedures and attitudes combined, coordinated and integrated into action, acquired through experience (formative and non-formative) that allows the individual to solve specific problems autonomously and flexibly in singular contexts" (ILO, 2008 quoted by Mulder, et al., 2008). Highlighting the main idea about the concept of competence studied by Mulder et al. (2008), it can then be defined that being competent refers to being able, having the ability to perform certain tasks, having the ability to perform a certain activity.
Competencies in Higher Education
Considering that the context in which the concept of competence in the educational environment is defined is relevant to justify the competency-based approach in the Design Module in the FUNIBER project area. In this sense, the proposals for competencies are based on a set of skills, knowledge and attitudes of a different nature, which include talents and intelligences (Cano-García, 2008).
It is important to mention that Goal 8 of the 2030 agenda developed by the International Labour Organization (ILO) states that it is necessary to “reduce skills mismatches through training programs that respond to the needs of the labor market and the introduction of work experience components in technical and vocational education and training”. (OIT, 2017, p. 13). As well as within its objective 4 oriented to Quality Education, it indicates that “by 2030, to substantially increase the number of young people and adults who have the necessary skills, in particular technical and professional, to access employment, decent work and entrepreneurship”. (OIT, 2017, p.23) These guidelines justify and frame the work of this research.
It can be analyzed that the capacity for critical analysis and the application of knowledge are different actions and involve a much broader dedication than just the acquisition of theoretical concepts. “For this reason, it is more necessary than ever to revise the pedagogies and didactic methodologies used, so that they are not focused on the transmission of information, but on the promotion of competences in individuals”. (Romero, 2010, p.90).
Mulder et al. (2008) then cite and analyze three approaches for the competencies studied by other authors such as Norris, (1991); Eraut, (1994); Wesselink, (2005); these approaches are the behavioralist approach, the generic approach and the cognitive approach.
The behavioral approach refers to competencies acquired through training and development, according to McClelland (1998), quoted by Mulder et al., (2008) are based on the description of observable behaviors or performances in situ. The characteristics of this approach are the demonstration, observation and evaluation of behaviors or behaviors. (Mulder et al., 2008)
The generic approach identifies common skills that explain variations in performance.
In this approach, statistical analyses can be performed to highlight the main and generic characteristics of the people who are most effective. (Mulder et al., 2008)
The cognitive approach focuses on competencies that include psychometric models of human intelligence, associated with the processing of everyone’s information. “These specialized competencies refer to a group of cognitive prerequisites that individuals must possess in order to perform well in a given area.” (Mulder et al., 2008, p. 4).
On the other hand, Tobón (2008) proposes to interpret the term competencies as “Complex processes of performance with suitability in certain contexts, integrating different knowledges” (p.5). When he refers to “knowledge” he highlights it as: “know how to be, know how to do, know how to know and know how to live together” focused on carrying out activities or solving problems.
Based on this vision, Tobón (2008) proposes the analysis of competencies based on six aspects associated with the concept from a complex approach applied to an educational environment: processes, complexity, performance, suitability, metacognition and ethics. “This means that in each competency there is an analysis of each of these six central aspects to guide learning and evaluation, with implications for didactics, as well as for evaluation strategies and instruments.” (Tobón, 2008, p.5).
Although the approach proposed by Tobón (2008) has not been taken into account for this research, it is considered relevant to present it as an antecedent and also to leave it as a future indicator for the comparative effects that will be carried out in the next section of the research.
Considering these descriptions, it is considered that within these three concepts this research focuses on the generic approach, as the variations in the different competencies will be analyzed through the study and analysis of the main and generic characteristics of the sample studied.
IPMA 4.0 Certification for Project Management
A project is the only result that is limited by the scope of time, cost and quality, involves a temporary effort and must be continuously adapted to the context in which they are carried out. (López-Paredes et al., 2010).
(IPMA, 2015) describes project management at the professional level as the discipline that should have standards and guidelines to define the work. It is essential to develop and standardize the skills associated with these requirements and that are used in project management by people. The ideal situation to carry out a project is that all people, teams and suppliers are competent to carry out their work.
A competency is a compendium of knowledge, skills, abilities and relevant experiences necessary to be successful in a given function. (IPMA, 2015). According to ISO/IEC 17.024, “a certification is the culmination of the process of formal recognition of workers' competences; it implies the issuance by an authorized institution of an accreditation about the competence possessed by the worker” (Márquez & Arzola, 2008, p.8).
Project Management Institute (2017) defines at a general level the competencies associated with the project manager, referring to the skills that must be used through the talent triangle proposed by PMI©, which focuses on technical project management, leadership and strategic and business management. These three axes are broadened and defined more specifically, addressing specific issues in the IPMA 4.0 certification, focuses on the development of personal, technical and contextual skills that allow the project manager to face a comprehensive view of the problem. Considering that this research is based on the evaluation of the level of competences defined by IPMA 4.0 for a project manager, in the students who take the Design Module in the FUNIBER project area, it is appropriate to briefly mention the competences defined by IPMA and the structure of the Design Module followed by the students of this program.
IPMA (2015) points out that a project manager should have an overview of all components of the project, program or portfolio. He has therefore divided the skills associated with project management into three groups: skills associated with people, skills associated with perspective and skills associated with practice.
The competencies associated with people include: personal integrity and trustworthiness, personal communication, motivation and commitment, leadership, teamwork, conflict and crisis, initiative, self-confidence, negotiation and results orientation.
The competencies associated with the perspective include: strategy, governance, structure and processes, standards and regulations, power and interests, and culture and values.
The competences associated to the practice include: project designs, programs and portfolios, goals, objectives and benefits, scope, time, organization and information, quality, finances, resources, provisions, planning and control, risks and opportunities, interested parties and change and transformation.
In total, they are 28 competences in which IPMA points out that a project manager must have knowledge, abilities and skills to ensure the success of a project.
About the Design Module in the Projects Area
Tobón (2008) mentions that the most relevant strategies to implement a module at an educational level are: the project method, the simulation, the learning based on problems, the maps method and the Kolb constructivist method. It presents the modules as plans which integrate learning and assessment with the aim that the student develop competences, enabling in this way to ensure that the student’s profile is in line with the competences needed in the field in the program they are enrolled in. The analysis, the understanding and resolution of a problem which is detected and associated with the thematic the program proposes is necessary to establish the module. It is pertinent for this research to mention that for Tobon (2008), the module has not only the purpose of learning of the students but the permanent reflection and improvement by the teachers when facing real cases which might require new strategies or views.
The methodology used by FUNIBER in the Module for the Master in Project Design, Administration and Management is focused on the integration of theory and practice, considering that it must connect activities focused on both a theoretical and practical level around real problems previously detected to develop, through theory and practice, the development of competences associated to project management.
The contents of the teaching materials of FUNIBER’s Design Module is divided into nine exercises which are developed progressively. Each exercise is revised and assessed by a tutor which orients the progress of work. Work is made in teams formed by groups of between 4 and 6 students from different nationalities which work in a virtual collaborative working environment to develop the Design Module. All the exercises have theoretical material which supports the contents, theories and units associated to each exercise. Below we are going to describe briefly the area which address each exercise developed in the module.
In exercise one (1), the objectives which are pursued are aimed at the understanding and definition of the problem and conflict, introduction to the concept of project and the presentation of the problem-solving process (FUNIBER, 2017a).
In exercise two (2), the aims pursued are aimed at identifying the final product and the operative unity in the project, recognizing the different kinds of plaintiffs involved in the project and their relationship in the system to be implemented (FUNIBER, 2017b).
In exercise three (3), the objectives are to identify the requirements of the relevant involved parties, identify the constrictions and restrictions of the project and relate them with the parties involved; and finally to describe the characteristics of the service to be provided to meet the needs of the involved parties, considering the existing limitations (FUNIBER, 2017c).
In exercise four (4), the objectives are to learn the methodology to carry out the identification and organization of the processes which take place in a project and to know the useful tools to formally document a process map and a working organizational chart (FUNIBER, 2017d).
In exercise five (5), the objectives are to identify the variables which influence the working conditions in order to consider them in the creation of jobs, to know the relationships between the jobs and the efficiency of the system, as well as the relationship at a satisfaction level of the person who becomes employed. To think about the characteristics related to the people which will be employed and recognize the relationship between the working conditions and the human reliability presented by the person who becomes employed in a certain work (FUNIBER, 2017e).
In exercise six (6), the techniques of analysis and identification of risks for the project are made known, knowing to quantify the risks and plan responses to them, introducing the concept of sensibility and tolerance of risks and recognizing the appropriate tools for taking decisions facing risks (FUNIBER, 2017f).
In exercise seven (7), some basic concepts about economic assessment of projects and tools to formulate a budget framework and an inversion schedule are introduced (FUNIBER, 2017g).
In exercise eight (8), the basic concepts about the phases of the life cycle of a project and the essential characteristics of each phase are recognized, and the importance of the organization between the formulation and management of the project phases are assessed (FUNIBER, 2017h).
In exercise nine (9), a summary and presentation of all the work made in the previous exercises are made.
The development of the Design Module is carried out meeting a delivery and revision agenda by the students and tutors respectively, in which there is a feedback about the development of the proposal of a project, following the structure which as previously described.
In this research, the design is non-experimental, because the study variables have not been manipulated intentionally, it is exploratory in its initial approach because it pretends to know the level of self-assessment which the sample selected has about a specific theme, without deepening in the relation between variables, with a mixed approach because, although the indicators of the competences are described qualitatively, a quantitative interpretation of them is made.
The sample analyzed for this work is 36 students selected randomly within a population of 200 students which started the Design Module in the projects area in May 2018.
The measurement tool has 29 questions validated internally with the Index of Cronbach´s Alpha and externally validated by experts in the field. Each one of the 28 competences which IPMA points out in its 3 areas are associated to a question, except the teamwork competence which was associated to 2 questions, considering that making this question about virtual environments of ream work is appropriate in the online context in which the Design Module is developed.
In the application of the survey, the tool Google Forms was used to distribute the tool and collect the data. In the statistic projection of the results, Unscramble 10.3 was used to analyze the main components, and Excel 2010 for the dispersion graphics.
The results obtained are represented in two formats. In both cases the self-assessment levels described in the surveys were synthesized at only 3 levels. For this purpose, in the cases in which there were 5 or more self-assessment levels, the values are grouped as seen in the table 1.
Description of the criteria used for reducing the answer variability to only 3 levels.
|Answer options||Interpretation for analysis of results|
|Assessments between 1 and 10, being 1 Deficient and 10 Excellent||For this case, answers between numbers 5 and 10 were only obtained. The values of the sum of the percentages obtained 5 and 6 at Low level, 7 and 8 at Intermediate Level and 9 and 10 at High Level.|
|Perfectly Capable, Very Capable and Averagely Capable||Averagely Capable Low Level, Very Capable Intermediate Level and Totally Capable were considered as High Level.|
|Always, Many Times and Sometimes||Sometimes was considered as Low Level, Many Times was considered as Intermediate Level and Always was considered as High Level.|
The first results format is a projection of all the competences included in the three areas proposed by IPMA regarding the distribution of levels of answers with a greater frequency in the sample studied (Figure 1).
Figure 1. Projection of the competences per area based on the distribution of answers.
Note: Source: Author’s creation, 2018.
In Figure 1, the results of an analysis of Main Components (PCA) (Abascal & Grande, 2005) regarding the distribution of answers with maximum frequency on Intermediate Level is shown, provided that the tendency which explained the greater variability percentage in the competences studied (68%). The remaining percentage is explained by distributions which have high frequencies for the self-assessment levels Low and High. This graphic shows that, for the sample studied, most of students assessed themselves with an Intermediate Level in 92% of the considered competences. The other 8%of competences correspond to two competences in the area of people: Integrity and Results Orientation. For Integrity, a distribution with a greater results frequency in the High Level was found (projection in blue located at -9 units from the second main component and at 59 unities from the first main component), and for the Results Orientation, an almost equitable distribution between Low and High Levels (projection in blue located at 46 unities from the second main component and at 45 unities from the first main component).
The second format in which results of this study are presented is scatter charts, specific for each one of the areas described by IPMA (Perspective, People and Practice). The percentages of answers obtained in each self-assessment level are detailed for each considered competence (Figures 2 to 4).
In the case of the Perspective area (Figure 2), it is seen that, for the sample studied, the greater answer percentages are located in the Intermediate Level, with values over 50% and the tendency in the 5 competences is the same in all cases.
Figure 2. Distribution of answer percentages for the competences in Perspective area.
Note: Source: Author’s creation, 2018.
In the case of the area of People (Figure 3), for the sample studied, a similar tendency to the one found in Perspective for 4 competences was found: Initiative and Inventiveness, Negotiation, Teamwork and Leadership. The competence of Personal Communication had an answer percentage of High Assessment level inferior to the one observed for the tendency previously described. On the other hand, the answer percentage of Intermediate self-assessment is superior to 70% in this case. Finally, the two extreme cases found in this are Integrity and Confidence, whose answer frequency was over 80% and the Results Orientation, whose percentages were distributed almost equitably between Low and High levels (48% and 505 respectively).
Figure 3. Distribution of answer percentages for the competences in People area.
Note: Source: Author’s creation, 2018.
In the Practice area, the formation challenge (the goal or the greater difficulty for the professor) is evident is the competence of Change and Transformation, where the self-assessment answer percentage in High level was the lowest of all the studied competences (inferior to 10%). Simultaneously, it can be seen that the Project Design competence has the greatest frequency of assessment in High level in this amount of data. The other competences follow the tendency previously observed, with a maximum answer frequency in the Intermediate level, with percentages between 40% and 80% of the answers collected.
Figure 4. Distribution of answer percentages for the competences in Practice area.
Note: Source: Author’s creation, 2018.
Providing response to the objective explained in this work, the students of the Design Module in project area which started the transversal course in May 2018 assess themselves with an answer frequency distribution with maximum in Intermediate Level with 92% of the competences studied, as well as a distribution with maximum in High level for the competence of Integrity and Confidence and the equivalent Low and High in Results Orientation.
To complete a low diagnosis, IPMA perspective recommends optimizing the measurement tool with competence self-assessment tools under different approaches. The optimization of this tool might be additionally taken into account from the perspective of complex approach proposed by Tobón (2008), who considers six assessment tools for each competence. In addition, for future diagnosis activities of the students´ level, it is proposed to use statistic told both for the selection of the sample to be studied and the preparation of the assessment levels of the questions included in the surveys (use of techniques of Experiment Design). However, to complete this work with data obtained at the end of the module, it is suggested to use the same measurement tool, the same student sample, as well as the same statistical analysis tools in this first diagnosis for the description of the results for comparative purposes. Obtain the data through this mean will be essential. However, this fact does not restrict the possibility to use simultaneously a new optimized tool on the basis of the criteria described in the previous paragraph. By this way, two objectives might be reached: to assess the actual state of the Module of Design and to optimize the competences measurement tool.
Abascal, A., & Grande Esteban, I. (2005). Análisis de encuestas. Pozuelo de Alarcón, Madrid : ESIC Editorial. Retrieved from http://cataleg.uab.cat/iii/encore/record/C__Rb1931136__Sanalisis de encuestas__Orightresult__U__X7;jsessionid=D3971750281BE6031D4250AD2C4C8C5A?lang=cat&suite=def
Cano-García, M. E. (2008). La evaluación por competencias en la educación superior. Revista de Currículum y Formación Del Profesorado, 12, 1–16. doi: 10.1016/j.neuropharm.2007.11.003
FUNIBER. (2017a). Módulo Teórico 01, Master en Gestión de Proyectos. Argentina: FUNIBER.
FUNIBER. (2017b). Módulo Teórico 02, Master en Gestión de Proyectos. Argentina: FUNIBER.
FUNIBER. (2017c). Módulo Teórico 03, Master en Gestión de Proyectos. Argentina: FUNIBER.
FUNIBER. (2017d). Módulo Teórico 04, Master en Gestión de Proyectos. FUNIBER.
FUNIBER. (2017e). Módulo Teórico 05, Master en Gestión de Proyectos. Argentina: FUNIBER.
FUNIBER. (2017f). Modulo Teórico 06, Master en Gestión de Proyectos. Argentina: FUNIBER.
FUNIBER. (2017g). Módulo Teórico 07, Master en Gestión de Proyectos.
FUNIBER. (2017h). Módulo Teórico 08, Master en Gestión de Proyectos. Argentina: FUNIBER.
IPMA. (2015). Individual Competence Baseline for Project, Programme & Portfolio Management (4ª Ed.). Zurich: IPMA.
Latorre, A. (2003). La investigación - acción Conocer y cambiar la práctica educativa (1ª Ed.). Barcelona: Editorial GRAÓ.
López-Paredes, A., Pajares-Gutiérrez, J., Galán-Ordax, J. M., Práctica, B., Industrial, I., Certifi, I. – C.,Galán-Ordax, M. (2010). Organización y dirección de empresas La empresa como cartera de proyectos y programas. Dyna Febrero, 85, 1–39.
Márquez, M., & Arzola, M. (2008). Análisis comparativo de los sistemas para Certificación en Gestión de Proyectos otorgados por IPMA , PMI , AIPM y PMAJ. In II International Conference on Industrial Engineering and Industrial Management XII Congreso de Ingeniería de Organización (pp. 93–104). Retrieved from http://adingor.es/congresos/web/uploads/cio/cio2008/BUSINESS_ADMINISTRATION_AND_STRATEGY//93-104.pdf
Mulder, M., Weigel, T., & Collings, K., (2008). El concepto de competencia en el desarrollo de la educación y formación profesional en determinados países, miembros de la UE. Profesorado: Revista de Curriculum y Formación Del Profesorado, 12(3), 7. doi: 10.1016/S2215-910X(14)70038-2
OIT. (2017). El trabajo decente – Crucial para el desarrollo sostenible. Ginebra. Retrieved from http://www.ilo.org/global/topics/sdg-2030/resources/WCMS_470340/lang--es/index.htm
Project Management Institute. (2017). La guía de los fundamentos para la dirección de proyectos (Guía del PMBOK) / Project Management Institute. (6ª Ed.). Pennsylvania: Project Management Institute, Inc.
Romero, M. (2010). Demandas Formativas. Revista de Antropología Experimental No 10, 2010. Especial Educación 8: 89-102. Universidad de Jaén (España) ISSN:, 10(ISSN: 1578-4282), 89–102. Retrieved from http://revista.ujaen.es/rae
Tobón, S. (2008). La Formación Basada En Competencias En La Educación Superior. Universidad Autonoma de Guadalajara, 1–30. Retrieved from http://cmapspublic3.ihmc.us/rid=1LVT9TXFX-1VKC0TM-16YT/Formación basada en competencias (Sergio Tobón).pdf
Zabala, A., & Arnau, L. (2008). Cómo aprender y enseñar competencias (2ª Ed.). Editorial Graó.