1. Introduction

The question of formation and its impact on the organization is increasingly becoming one of the central aspects in the reflection concerning the professions of help and assistance. However, more than in the past, it is not only a "reactive" problem, that is, of adaptation to regulatory requirements or to the evolution of organizational systems, but a cornerstone in the development of human resources.

Training in health is a form of systematic and daily commitment, aimed at ensuring a satisfactory quality service for citizens. In this sense, this commitment outlines the properly ethical dimension of the construction of the different professions, or, if you prefer, of their "continuous training". In particular, the meaning that we want to attribute to training is that of a tool through which to promote the redefinition of roles and the growth of skills, evaluating in the medium and long term the organizational repercussions of the proposed training interventions.

In public organizations, specifically in health companies, the objective of improving the services rendered to the citizen is a priority, and it is through the enhancement of operators, the greater wealth available to the system, that can be achieved. Training is therefore of strategic importance as a policy for the development of individuals and organisations.

The general principle of prudence in investment, in this age characterized by the scarcity of resources, represents an ethical necessity, to ensure that the limited availability is allocated, even in the field of training, in the most efficient and effective way possible.

From a survey carried out at regional level in the early 2000s, the Civil Hospitals of Brescia turned out to be the hospital with the largest number of Technical Operators in charge of Assistance to be retrained as since the 80s the training had believed in the role of intermediate figures of assistance, partially retraining the Specialized Social Health Auxiliaries and training a total of about 500 people. 76% of these operators were then retrained in Social Health Workers over six years. For the Lombardy Region, the establishment of the profile of the Social Health Operator in 2001 and the definition of the training course in 2007, was the premise not only for the start of the retraining courses but above all for the (re)definition of the skills of the different profiles that intervene in personal care,  starting with the nurse, responsible for care planning.                                                                                    

With the regional plan of interventions to enhance the health professions and support figures, the Region has identified in the area of personal services, as a priority objective of the health sector, that relating to the "qualification and enhancement of human resources and professionalism" in response to the need for humanization of assistance on the one hand, and of ever greater quality of assistance on the other. All this in line with the most advanced Europe.

In accordance with the assumptions highlighted above, the specific experience of behavior analysis and evaluation is reported, which represents an extract of a broader research work on the effectiveness of training (Salas and Cannon-Bowers, 2001), aimed at investigating the direct and indirect effects on post-formation behavior of some personal and contextual variables. [1]

Within this framework, the analysis of behavior has represented one of the most important evaluative components in terms of operationalization, as well as the theoretical and scientific value of the work carried out.

2.Theoretical framework

        In organizational contexts, the evaluation process of training focuses mainly on outcomes such  as reactions to training activities and learning, understood, the latter, as the acquisition of declarative knowledge. On the other hand, the size of the service is less investigated both at the end of training and at work (Van Buren and Erskine, 2002).

In science, an opposite trend is observed. The work of Colquitt, Le Pine and Noe (2000) highlights how the most investigated outcomes are those related to declarative and procedural knowledge and the study by Arthur, Bennett, Edens and Bell (2003) shows that only 4% of the contributions examined by their meta-analysis work report data on reactions compared to 59% for knowledge and 31% for behavior.

The examination of the literature on the transfer of training shows how several studies highlight the need to verify behavior at work and above all to address this analysis after a certain period of time (Brown, T., 2005). This is so that the person actually has, from a temporal point of view, the opportunity to apply at work what he has developed through training. Reference is also made to how complex this aspect can be, especially in organizational contexts, where opportunities to apply the acquired knowledge at work do not always arise, becoming impossible, consequently, to examine the process of generalization (Herteistein, 2001).

Based on the transfer of training models  in the literature, it is essential to have a measure of behavior to understand the contribution of training (Ellis and Davidi, 2005). The latter allows you to analyze whether the specific objectives expected on the context have been achieved, that is, whether people actually put into practice and act what the training has allowed them to develop. In view of the possible approaches to evaluation, as well as the objectives that the training has set itself, it should be emphasized that the emphasis placed on behaviour as an evaluation criterion is linked to the fact that a definition is associated with training that enhances the importance of the application on the work context of the knowledge acquired. It is evident that if training is seen as strictly instrumental to work performance, the same evaluation action must be carried out in order to grasp the elements of continuity between the two contexts. The behavioral component can be an element of reflection useful for evaluation and training planning, without denying the need to have a wider range of evaluation criteria (Colquitt et al., 2000) that are able to grasp the complexity of training and its effects on people and organizational contexts, especially when considering the broader meaning that also sees it as a political tool,  managerial and managerial (Goguelin, 1995; Bartlett, 2001; Loi and Lemoine, 2010).

Focusing on the tools related to the detection of behavior for evaluation purposes, it was shown that performance measurements are more reliable if the description of behaviors takes place in a specific way (Tziner and Kopelman, 2002) and if the action to be evaluated is described as well as how it can really happen at work. The authors compare different types of performance assessment tools including grids, Behavior Observation Scales (BOS) and Behavioral Anchored Rating Score (BARS) with respect to the emotional response of the people evaluated. While the grids are represented by simple scales of scores that give a generic indication with respect to the individual performance, the BOS and BARS report the performance dimensions to be evaluated with the relative score. The difference between the two types is that the BOS in addition to the performance dimension also reports the work behaviors associated with it, which are actually carried out in the workplace. In addition, the score scales indicate how much a behavior is implemented in a well-defined time frame. The authors pointed out that evaluations carried out with the BOS typology are associated with greater satisfaction on the part of the evaluators. This satisfaction is characterized by a lower perception of the level of anxiety relative to the evaluation.

These results suggest that, where BOPs are indeed necessary in relation to the objectives of the evaluation, they would be preferred over other types of instruments precisely in view of the positive effects they have in the evaluators.

In line with this perspective, within the work carried out in the health context that will be described below, behavior at work was analyzed through specific behavior sheets similar to BOS as one of the criteria for the evaluation of training effectiveness. To this end, we started from a specific description of the activities to be carried out within the work context for its operationalization.

3. Methodology

The training course covered by the evaluation involves a single professional figure, such as that of the Technical Assistance Operator (OTA), following which the qualification of Social Health Operator is expected to be issued.

The figure of the Social Health Operator (OSS), described by the national resolution and taken up by that of the Lombardy Region, in the hospital has the function of assistant nurse. The main objective no longer concerns the hygiene of the premises (increasingly outsourced function), but the care of the patient, under the responsibility of the nurse. This operator, in fact, deals with nutrition, personal hygiene of the patient and his mobilization, according to the criteria of low discretion and high standardizability of the tasks assigned. In addition, it assists the nurse in the administration of basic therapies and in the detection of some vital parameters, such as the control of body temperature and blood pressure.

The course, lasting 130 hours, included three weekly meetings (for about three months) and a week of internship at the end of the training activity, lasting 50 hours, at a department or garrison other than that of the operator. It should be noted that the duration of the training course through which to obtain the qualification of OSS varies according to the previous title held by the student. For those who have no training in the field, this qualification is achieved as a result of a training course of 1,000 hours.

The programming of the various training modules, carried out according to the cognitive background of the participants, focused essentially on three macro areas of intervention, such as: the role of the social health worker, the specialized technical bases and the relationship with the user. The teaching team remained identical for all the courses carried out, as well as the examination board.

The achievement of the OSS qualification took place following a theoretical and practical examination aimed at assessing its declarative and procedural knowledge. For the purposes of the qualification, the evaluation expressed by the internship manager was also examined.

Participation in the course was voluntary and took place during working hours. The only selection criteria followed concerned the qualification and a minimum of work experience (2 years). At the end of the course, the hospital company has provided for the conversion from OTA to OSS of all qualified people.

Prior to the start of the research, the aims, objectives and procedure of the research were presented to the company managers and the various department coordinators through meetings organized to illustrate the objectives of the training and the rules for organizing working hours. The coordinator, once informed, then forewarned his collaborators of the ongoing research.

3.1  Research design

The overall research adopted a sequential longitude perspective with a pre-experimental research design as it involved a pre-test and a post-test without a control group.

A self-administered questionnaire was used and an external evaluation carried out by the department coordinators was used for the analysis of the behaviour of each participant, which also took place before and after the training.

The behaviour detection instrument was administered in two measurement times: at the beginning of the training (T1) and twelve weeks after the end of the training (T3). The expected period to carry out the survey at a distance of time from the training is based on previous research (Chiaburu and Tekleab, 2005), and on the criterion of adequacy with respect to the objectives of the training and organizational needs. In this case, the Company Spedali Civili in which the research was conducted has introduced in its strategic objectives the conversion into OSS of all the operators present in the organic endowment who have acquired the new qualification, functional to the organizational redesign determined by the new care models and the workload of the nursing staff.

Each participant to fill out the questionnaire anonymously was able to choose a personal code of five digits and / or letters with which to identify, which was also used by the coordinators to carry out the hetero evaluation. Once completed, the questionnaires were sent to the Training Office through the internal mail service and subsequently delivered to the researchers.

3.2 Tool
Pre and post training behavior.

To describe the activities carried out by the Social Health Operator, the professional profile designed by the Regional and National regulations was used. These are all the tasks that this professional figure must perform in order to fulfil his function as assistant to the nurse.

In order to have a more precise and more representative description of the work carried out within the hospital reality, interviews were conducted with the coordinators who verified the adequacy of each task in describing the overall activities carried out by the OSS. These interviews allowed the construction of a card of work activities actually carried out by the SDGs. The list of behaviors was built following the indications given by Tziner and Kopelman (2002) for the construction of the Behavioral Observation Scale and consists of 15 activities. The latter represent the focus of the activities that a social health worker carries out within his department.

Each course participant was asked to specify to what extent each behavior described represents an activity performed or not, the answer of which is based on a five-point scale where 1 corresponds to "not at all true" and 5 to "absolutely true".

The average of the scores was adopted to define the extent to which the person actually applies the behaviors required by the OSS after carrying out an exploratory factor analysis, which allowed to detect the presence of a single factor. The Cronbach alpha coefficient was calculated to verify the reliability of the measurement (T1α=.88; and T3 α=.93).

The following statements are some examples of items: "I welcome the user and his family trying to facilitate their insertion by orienting them in the various services of the department", "I take care of detecting risk situations to refer them to the health personnel", "I favor the patient's walking and posture changes".

The same tool was also adopted by the coordinators to express their assessment in both Q1 and T2. However, for the hetero-evaluation, it was decided to consider only the post-test, as the forms filled out at T1 were small. The reliability index of the tool is α=.88.

Declarative knowledge.

The latter represent the cognitive elements possessed by people and refer to what they know about the topics covered in formation (Kraiger, Ford and Sals, 1993).   Within this research work the latter have been considered in order to verify that people have learned the contents addressed. They were evaluated through 22 multiple choice questions adopted in previous exams aimed at evaluating candidates for the qualification of Social Health Operator. They were recorded at T1 and at the end of the training activities (T2). To define the score, the average was calculated using the correct answers.

3.3 The champion

The sample that was reached in the first survey is represented by 164 auxiliary technical operators, while in the follow-up survey  (T3) 105 questionnaires were returned. However, there are 85 complete questionnaires that could be used for the analyses. These represent those who responded to all three surveys and make up 51% of the entire sample reached with the first survey. It should be noted that the professional qualification was obtained from the entire sample considered. The majority of the sample is female (87.1%) and the average age is 41.09 (DS= 6.05) with an age range ranging from 28 to 52 years. As for the qualification, 57.6% have a lower secondary school license, 25.4% have a high school license and 15% have a vocational school diploma. Most of the people involved have a full-time permanent contract (83.5%), only one person is hired on a fixed-term basis and 15.3% have a part-time contract. 50.6% of the sample has been working in the hospital for at least five years and no more than 10, while 8.2% have been working for less than five years, 31.8% have been working for more than 10 years, 9.4% have been working for more than 20 years, while there is no one who is close to retirement and who has been working within the structure for less than a year. Altogether, most of the departments present in the structure are present with the exception of infectious diseases.

4. Results

For the analysis of the data, averages, correlations were taken into account and the analysis of variance was carried out in order to examine the evolution of the evaluation criteria between before and after training. In particular, Table 1 shows for each variable the averages, the standard deviation, the reliability indices of the scale. With regard to the property of normality, the indices of asymmetry and curtosis are reported, which will be particularly useful for the analysis of declarative knowledge.

The averages show that after some time from the end of the course people believe they apply the characteristic behaviors of the professional profile of social health workers and in part, they believe to apply them even at the beginning of training. This fact is explained by the fact that these figures have been operating for some time within hospital departments, already carrying out basic care activities, albeit with minimal responsibility.

In relation to the knowledge it is noted that at T1 the average of the correct answers is equal to M = 0.69 DS = 0.10 and at T2 is M = 0.85 DS = 0.11. It also emerges that at T2 both the asymmetry and curtosis indices are not below those that are considered the limits within which the analyzed variable can be considered normal (Barbaranelli, 2003). This means that the responses tend to be, in this case, particularly positive. In the literature it is possible to find the same tendency also in the work of other authors. Tracey, Hinkin, Tannenbaum and Mathieu (2001), in particular, finding high indices of asymmetry and curtosis, attribute the cause to the simplicity of the test. Indeed, even in this case, if we analyze in more detail the frequency of positive responses we find that the correct ones represent 85% of the answers provided to the test (the authors mentioned above in their work find that 83% of the answers provided are correct). This data suggests that the test may have been excessively simple and consequently, calls for caution in using the data for evaluation purposes.

As far as the correlation table is concerned, it is to be considered particularly useful as it allows to show that the behavior evaluated by the managers correlates with the self-evaluated one.

This aspect leads to believe that heterosexual and self-evaluation, having a positive relationship, agree in the detection of the behaviors actually applied.

Another relevant fact is that declarative knowledge does not correlate positively with behavior.  It is important to note that this is not entirely new in the literature on the transfer of training. In the meta-analysis of Colquitt et al. (2000-2014), for example, it emerged that declarative knowledge has a generally moderate influence on transfer and how this may depend on the low reliability of the measures used, such as Tracey et al themselves. (2001) found in their own research work.

A further aspect that was analyzed is the existence of differences in the values of the evaluation criteria between the pre-test and the post-test, which translates into a significant increase in the knowledge acquired and the behaviors applied at work.

The research design used, almost experimental, does not allow to affirm tout-court a cause and effect relationship between formation and change. It is to be considered adequate (Sackett and Mullen, 1993), however, to verify whether people have achieved a set of objectives, such as knowledge, skills and attitudes functional to the performance of their work and which are closely related to training.

For the analysis of the differences between pre and post tests, the variance analyses were used, the results of which are shown in Tables 3 and 4. They show that both in relation to the knowledge acquired and in relation to the behavior there is a significant difference between before and after training. In relation to this result, however, it should be noted that the size of the effect is greater for declarative knowledge while it is minimal for behavior.

5. Discussions and Conclusions

    The objective that this contribution has been achieved is to illustrate a part of the experience of evaluating the training effectiveness of a professional qualification path, focusing mainly on the aspects of evaluation of behavior and declarative knowledge. The procedure through which the pre- and post-training behavior analysis tool was defined in particular and through data analysis, it was possible to highlight some problems related in particular to tests on declarative knowledge.

In summary, the pre- and post-test research design made it possible to highlight an evolution with respect to behavior and knowledge between first and post-training.

The analysis of the data has made it possible to highlight how knowledge tests can prove inadequate to reliably detect how much people apply new behaviors at work or not. It may, therefore, be useful in the planning phase of the evaluation to reflect on the usefulness of the verification of knowledge. It may be adequate to understand part of the cognitive aspects related to training, but it may not be as adequate if you wanted to predict how much the person will use what they have learned in their work (transfer of training).

This aspect is closely linked to the way of conceiving training, to the functions attributed to it with respect to the person and to work (Lemoine, 1993). How much is training a tool for the growth of the person? How much for the quality of the work? How much for the organization? The choice of evaluation criteria, together with the search design allows you to set the type of answer to these questions.

The perspective used in this work draws its origin from the models of evaluation of training effectiveness centered on the concept of transfer of training, understood more recently as a process of generalization of the knowledge acquired in training in work contexts, where the analysis of behavior acquires a relevant value in the evaluation process. In this context, training is intended as a tool for the acquisition of a set of knowledge-skills useful for the realization of the work (Ellis and Davidi, 2005). The research work that has evolved in this field has therefore tried to understand what inputs would allow to facilitate this process of generalization, without questioning the very concept of training adopted, together with its multiple functions for the individual, for work and for the organization. What is interesting to note is that today this approach is giving way to a more complex idea of training that transfer of training research  is integrating into its evaluation models (Mathieu and Tesluk, 2009).