Determining Validity of Research Framework

Determining hardship of Research modellingCHAPTER SEVENFRAMEWORK VALIDATION7.1Introduction This chapter focuses on the evaluation and validation of the mannikin. The query severity and manner deployed to maximise the boldness will be discussed in the first part of this chapter. Also, the eternal modeling validity will then be discussed. The second part of this chapter will analyse and discuss the results obtained from the evaluation of the exemplar. The example evaluation has been carried pop verboten by academics and rights practitioners and has pass judgmentd the suitableness and impellingness of the material in the construction sector.7.2 bindingityThe validation process started at the first stage of the enquiry, making sure that the research participants, eyeshade instruments and condition were exclusively accomplishing the research aim. A way come to the fore of writers learn contended that validity dissolve non be applied to qualitative research. At the similar time, demand understood the need for some qualifying check or measure for their research (Golafshani 2003). Cl early there atomic number 18 no direct tests validating qualitative research however, this does not c totally for that there atomic number 18 no guidelines (Patton 1990). Through the stages of this research, the effort has been make to contract that (1) the measuring instrument is measuring the concept being referred to and no other(a) concepts and (2) there is an accu array measurement of the concept (Bailey 1994). The measuring instruments apply in the interview method were the interview questions also the method used in analysing the data salt away and building the fabric. therefore, it was important to ensure that all research propositions were clear and effectively measured. Also, the method of analysis ask the provision of right inferences.Qualitative research has often been portrayed by flexibility and uncertainty, and tension amidst creativity and rigour. therefore at the interview analysis and the stages of framework building, a critical compromise was cautiously considered (Pyett 2003). The analysis of interviews engaged a deductive process whereby the author was his or so trenchant critic (Cook and Campbell 1976, p 229). This is known as internal validity. however, in this fill, an open viewpoint of validity was used whereby different techniques heighter from Decaturen the analysis and understanding of the consecrate of fellowship gravel and retrieval and prompted more reliable and valid information of the framework (Golafshani 2003). These methods includedAn in-depth revision of the instruments used for data collection and the method of analysisCarrying out a peer review of interviews themes and methods of analysisCarrying out a pilot study assessing the language of the questions definitions and terms clarity of the purpose of questions for interviewConstruct a descriptive and relational framework for the interview analysis to unclutter sure there is the consistency of analysis criteria and accu assess interpretation and inferences.7.3 External ValidityDeveloping a framework for the capture and retrieval of WLC fellowship in a construction project is found on a small number of chemises, and does not imagine this framework can be suitable and effective to other construction boldnesss practising WLC in the construction sector. even so, researchers are also interested in the ecumenicisation of the findings beyond the investigated cases (Schwab 1999). The justification for carrying out an in-depth interview is that people tangled in a matured practice have knowledge that would somehow or another not be accessible to the researcher. Therefore, it is the quality of the understanding that is vital, instead of the number of respondents who share it (Wainwright 1997).While re inquisitive a few cases may accomplish excellent validity by giving a significant understanding of the pract ice in those cases they have been broadly criticised as miss external validity (De Vaus 2001). The generalising of findings to or across a target population sample is known as external validity (Pedhazur and Schmelkin 1991). There are no efficient or verifiable methods to carry out external validation based solely on a single examination of a research relationship (Schwab 1999). However, a definitive way to assess the validity of the proposed framework is to test such principles in reality, yet in spite of appearance research practice, this is rarely conceivable (Pyett 2003). fabric testing by the application method, it could suck in quite a while before the reasonable judgment of the framework validity could be r for each oneed. no matter of the possibility that this was to take a short time, there would be a need to access to a huge sample (representing the industry) and the framework implementation in this sector. However, in this particular case, this was viewed strong. Nev ertheless, a few other methods were able to amplify the framework validity in this study as outlinedSelect fount case studies organisations (of current and best practices) from the target population in the interview stage.The pilot study provided a solid ground for understanding the current practice of KM and and so it was utilised as direct principles in using the best practice to suit the industry, which boosted the generalising or external framework validity.Looking for the evaluations of the suitableness and posture of the framework in the construction sector by seriouss practitioners and academics.7.4 Framework evaluationTesting the general framework feasibility by applying it to a large number of organisations representing the construction sector have been difficult in this research. The framework evaluation by experts was viewed as another alternative for evaluation. in fact, this method was deemed as suppress with regards to risk, finance and value.Assuming that 100 r andomly selected organisations are a representative sample and the elections unavoidable are accessible in those organisations, it would take six months for the testing and implementation of the framework (practically it would take years). However what if the framework did not work? This would typify lost time of 600 months (100 organisation X 6 months) and therefore significant effort. Therefore, in this case, the evaluation method by experts could be deemed the more or less appropriate method as yet though the generalise by implement method was deemed possible.The reason for the evaluation method accordingly was not to anticipate that the framework was suitable for the organisation of the inscribed experts to obtain the experts evaluation in regards to the frame plant suitableness in the industry. The suitability of the framework was not seen adequate however because the suitable framework did not necessary means it would fetch value. Therefore, effectiveness was another c riterion the evaluation sought to answer.7.4.1 ParticipantsJustificationBased on their experience, the suitability and the effectiveness of the framework can be judged by experts by using the right tool techniques, required resources and methods. The time and effort required for the framework implementation can also be estimated and matched with the current practice.The participants involved in the framework evaluation is made up of two groups of experts academic and practitioners. The practitioners are involved with WLC practice in the construction project and therefore have a better understanding of the daily work. Also, their participation in a project which requires WLC practice means they can work with other organisation who practice WLC in the construction sector and can understand the verity of work environments.Also, academics are expected to have a deep understanding of different methodological viewpoints, as they know around observation and analysis. Further, the academic s rely greatly on the construction industry as a major source of information, so does not suggest that they will be isolated. Also, many academics are involved in the practice.SampleA total number of sixteen experts were divided equally between the academics and the practitioner who participated in the framework evaluation. The participants year of experience ranges from eight to thirty-eight years, accounting for the total of three hundred and thirteen years of experience. The eight practitioners were from the three case study organisations who participated in the pilot and main interview of the study. The focus on practitioners was on those in the plain stitch with experience in WLC practice because they had an in-depth understanding of the WLC practice. Also, participating academics were from four universities in the UK, working in the make environment and information and knowledge management departments.ExpertsJob titleYear of experiencePractitionersOrganisation A cypher Mana ger35 amount Surveyor20Building Surveyor7Organisation BQuantity Surveyor6Project Manager33Organisation CCost Estimator15Project Manager27Construction Manager7AcademicsSenior lector12Senior Lecturer30Lecturer6Lecturer7Lecturer25Lecturer25 lector31Reader21 turn off 7.1 Characteristics of participating experts in the framework evaluationBecause the framework evaluation required a seminar presentation, this process was thought to be long, and it was expected a low number of response. Therefore, it was unconquerable to reach the target of participants by sending invites to a member of staff with the three case study organisations. The total number invitations forwarded was 300 of which 120 were sent to academics and 180 to the three case study organisations.7.4.2 rule of EvaluationEmails was used a tool to invite experts and includedAn overview of the research projectRequest to participate in the evaluation of the framework for the purpose of validityRequest on the best times which s uits each of the case study organisation for a 40mins seminar presentationThe framework was presented to each case study organisation using Microsoft Office PowerPoint 2007. The presentation was interactive in which participants lessened the explanation while diagram part were moved in each presentation slides accordingly. The seminar presentation was arranged into ten slides which took a narrative era of logic since it began by clarifying the parts of the framework independently and after(prenominal)wards presented the strategic framework. puzzle out 7.1 A screenshot of the PowerPoint presentationThe questionnaire was designed and hosted using the online service provider Survey Monkey and consisted of two parts. Firstly, exposit about participants including name, organisation, job title, business and size of the organisation.secondly, the framework evaluation and this was included in two sectionsScore based questions to the rate the suitability and effectiveness of the develope d frameworkSeeking the opinions of the participants about the advantages, limitations and ways of improving the framework.It was discrete to present the framework diagrammatically showing the strategic, tactical and ope discerning levels. The presentation gave a more in-depth comment however diagrams were sent to four research students to assess their readability, and the outcomes affirmed that such diagrams could convey the concept and details of the framework if the reader had extensive experience in construction management. One of the research students who took part in the pilot study was a medical doctor, dissociated from the construction industry, who found the framework unreadable, Although he has a darling understanding of the major concept. Respondents were also sent an attachment of the presentations slides via email.7.5Results 7.5.1Suitability of frameworkParticipants were being asked to show their valuation for suitability for each of the components of the framework wh ere 1 implied unsuitable and 5 suitable. The answer from the analysis uncover that the levels of suitability for all approaches were considered far above the ground.As shown in Table 7.2, 3.74 out of 5 is the lowest mean value for the architecture of the knowledge base. Although it is not indicated in the value that the architecture of knowledge base is a 100% suitable, rather, it shows that it is likely going to be more suitable than unsuitable. The circumstance with regards to knowledge capture, retrieval and social organization approach much better as they all have a score of 4 out of 5. The skewness measure has a negative value which shows that the mass of distribution concentrates on the side of suitable. experience inhibitKnowledge retrievalKnowledge StructureArchitecture of Knowledge BaseNValid16161616Missing0000 stringent 3.93754.06254.00003.7500Std. variance .85391.85391.894431.00000Skewness -.605-.863-.639-.343Table 7.2 Suitability of the framework componentsFigure 7.2 shows that the majority of the participants gave a 4 or 5 rating of the suitability of all the framework components. With regards to the architecture of the knowledge base, the suitability was rated under 3 by two participants (i.e. 2). The normal curve illustrates that the approximation of the median(a) of all components almost falls in the 4 rate of suitability. All the indications provide turn up that those components are suitable for the construction organisations.7.5.2Effectiveness of framework Participants were being asked to show their rating for effectiveness for each of the components of the framework where 1 implied to be ineffective and 5 effective. The mean value of effectiveness for all components as shown is (Table 7.3) are above 3.75 and ranges between 3.87 and 4.18. The skewness measure has a negative value which shows that the mass of the distribution is concentrated on the side of effectiveness.Knowledge CaptureKnowledge RetrievalKnowledge StructureArchitecture o f Knowledge BaseNValid16161516Missing0010Mean4.06254.187503.93333.8750Std. Deviation.85391.910586.798811.02470Skewness-.863-1.019-.842-.571Table 7.3 Effectiveness of the framework componentsAs shown on the histograms shown in Figure 7.3 shows that the majority of experts participants evaluated all components to a score 4 or 5 rates of effectiveness. Knowledge retrieval gum olibanum turns out slightly more effective than the other framework components. The normal curve makes it clear the approximation of the average rate of all components which almost lies in the 4 rate of effectiveness. All the indicators together provide evidence that those components will be effective when implemented.7.5.3The Entire Framework SuitabilityExperts participants were asked to indicate their rate of suitability for the wide framework using the same criteria as mentioned above. As shown in Table 7.4, the mean value of suitability (3.93) and the negative value of the skewness measure provide an encoura ging indication for the external validity. In Figure 7.5, the pie chart shows that the majority of experts participants (62%) rate the suitability 4 out of 5 and the vast majority (81.3%) are either giving the framework a rate of 5 or 4. furthermore, the histogram in Figure 7.4 shows that sole(prenominal) one expert gave a rating of suitability under 3 (i.e. 2). Therefore, the indications of suitability are positive(p) and show that this framework is considered suitable.The Entire Framework SuitabilityNValid16Missing0Mean 3.9375Std. Deviation .77190Skewness -.881Table 7.4 The entire framework suitability7.5.4Effectiveness of the Entire Framework Expert participants were asked to indicate their rate of effectiveness for the entire framework using the same criteria as mentioned above. The mean value (4) and the skewness measure value (negative) which shows that the framework is evaluated to be effective as shown in Table 7.5. A third of the experts participant shows a full mark of effectiveness, and 40% scored the framework a rate of 4 for effectiveness as shown in Figure 7.7. However, as shown in Figure 7.6, three experts rate it a 3 and one a 2. Nonetheless, the normal curve is pointing to an approximation of effectiveness at the rate of 4. Indications show that the framework is effective.Effectiveness of the Entire FrameworkNValid15Missing1Mean4.0000Std. Deviation.92582Skewness-.623Table 7.5 Effectiveness of the entire framework.7.5.5Experts Opinion about the FrameworkThe expert participants were requested to express their opinion about the framework regarding suitability, effectiveness, advantages and limitations. This could give additional assessment not covered in the closed-ended questions. Twelve of the feedbacks were received and had been listed belowIt looks good to me. The problem is getting it work in practice. immense model and l like it, particularly the knowledge forms and the categorisation. This will help save time when searching for knowledg eI think the overall seminar presentation was good and it covered a lot of the things we do not consider as outlay pointing out in our day to day project livery implementation.It is very hard to make a judgment on these key issues just on the presentation. However, I am quite inspired by the approach to knowledge capture. Although the presentation is great, however, the information, it provides is insufficient.It seems very general.The framework works sanitary in suitability and effectiveness and in use of collaboration tools, intranet tools with knowledge management within organisations. It is essential for a large organisation that staff can find and utilise answers as well as gaining access to the subject matter, experts and sharing information on similar projects and programmes. The advantages are that knowledge is shared rather than lost or protected by individuals. The saying that knowledge is power is true of individuals However in organisations that are able and willing to capture knowledge and share this with the wider team rather than separate it with individual project members.The framework is suitable and effective in many areas and an insightful and logical bodily structure. There is some duplication, but that would seem to be unavoidable.Leadership support is important, with top-bottom commitment. Co-operation/sharing is not always a natural inclination in individuals.The framework seems to be super advanced and would require an IT system to invoke its processes and to encourage people to use it. If all members of the organisation are contributing positively to the framework and most of them have the ability to capture, retrieve, and then re-use the captured knowledge, then the system will be for sure highly effective. In reality, I think we share the knowledge sometimes, but even when we do so, we do not roll or capture them properly. As a result, we lose whatever we have shared as we cannot capture, structure and re-use all the knowled ge captured by only relying on our brain capabilities. We as practitioners need such a framework to assist us to have a better understanding of the knowledge so we can use it appropriately. From an overall point of view, I admire you for this excellent presentation, and it was informative and well structured. We have knowing a lot from this insightful presentation about knowledge management. There is no doubt that this framework is needed for business success.Although the framework appears to be complicated on its operational side I am certain that if the KM operation depends on this model, the result will be relevant and highly efficient.The framework makes sense from a project management perspective. However, knowledge capture and retrieval are complex and resource intensive activities and can be onerous to the project team if proper incentives are not adjust with the leaderships expectation of knowledge sharing.I think if your framework is properly applied, it will be extremely effective. Also, it is highly suitable. Regarding its advantages, I think it is simple to read and understand by any person who has a introductory level of knowledge management and its management.The issues mentioned in the comments can be grouped into four categories Firstly, those which highlight the agency of knowledge capture and retrieval.Secondly, those who stress the grandness and need for such a framework, due to the awareness of the problem, and thus confirm the motives for this research.Thirdly, indications that the framework may require a huge effort and construction projects already involved in demanding work. This issue has been discussed in the statement of the problem (section 1.4) whereby the problem lies on the capture and not the retrieval side. To minimise the problem, the project review was adopted as the main technique for knowledge capture which already exists in projects, rather than introducing a new approach. Additionally, the daily based reviews were rej ected in this research for the same reason. Furthermore, a strategy for motivation has been pinpointed in the framework to ensure the application of knowledge capture.Fourthly and most importantly, a suitable and effective framework will also require careful application and effort. This has been say by several experts, and therefore considered a critical point, because the good theory does not necessary guarantee successful application.Finally, an expert described the framework to be very general. As aIn fact the framework is considering a particular area to investigate, link to knowledge capture and retrieval of project management processes, in the context of the project. Models inside the framework have provided precise details regarding what to do in particular scenarios. Also, knowledge capture and retrieval cannot be tackled in isolation from issues such as.Leadership support and cooperative culture. It is worth mentioning that this expert selected the rating of 2 for all of the 10 rating-based questions which are considered statistically biased. However, it was thought that this academic expert provided his personal details and these matched the tracking information. Therefore it was considered a valid response, and his opinion was appreciated.Other points were also considered related to the advantages of the framework and impressions about the presentation.7.5.6 Framework ImprovementAnother question was put forward in regards to how the framework can be improved. The feedback from the expert participants are listed belowBy trial and error.The framework can be improved after applicationl did not analyse the presentation critically to make rational input in this directionI would rather read the whole PhD thesis and make my suggestionsWhat issues should be addressed? Profitability, time and teamwork?There is no such thing as ideal knowledge framework. Therefore the knowledge framework can also be improved. Evolvement of a framework can be made by lessons learnt from previous projects receiving constructive criticism and using good management techniques such as benefits and stakeholder management. Furthermore being clear about the benefits of knowledge capture and retrieval, with the onward transmittal/access of results by stakeholders, confabulation and using the framework communication channels for optimum use. The framework is excellent for understanding the needs and requirements of this age, and that knowledge should be shared for the benefit of all within an organisation rather than being kept and perhaps lost by the few, when the project team is dissolved or re-assigned.Show an example of how it will keep in line in practice.Working hand in hand with the IT system which enhances its process and records all captured knowledge after structuring it.The answers can be categorised into four major groupsFirstly, improvement is a continuous process, and the most appropriate way to improve the framework is through the application. Secondly, improvement can be achieved by addressing time, profitability and teamwork which have already been addressed in this study. Profitability is a significant factor that can declare the importance of knowledge capture and retrieval of the project and is, therefore, one of the areas to be embarked upon. However, the development of a model to measure economic profit alone is enough to do a research project.Thirdly, the exertion of effort in linking the IT system with knowledge capture and retrieval approaches. The knowledge base is a component of the framework, and there is a need to train people on the use of the knowledge base which links knowledge capture and retrieval approaches.Finally, the use of labels to distinguish between compulsory and optional activities and resources. This has been presented differently in this research as the majority of the activities and resources were thought to be important if knowledge was required to be captured and retrieved successfully. I t was decided only to underscore activities or resources that were suggested but not deemed necessary.7.6Summary This chapter has set out to determine the validity of the framework. It has demonstrated that validity relies on of the inherent validity of the methods of data collection, analysis, and the inferences led to the construction of the framework. Critical in-depth revision, piloting, peer reviews and a refractory framework for analysis were the methods used in this research to maximise its validity.For the purpose of generalising (external validity) the framework, a number of factors were considered early on in this project. These included using a randomly selected sample in the survey choosing a sample of best practice from the target population and comparing the results of best practice with the current practice obtained in the survey to assess the ability of and utilise the solution for the target population. Further steps were taken to test the generalisability of the framework by inviting practitioner and academic experts to evaluate its suitability and effectiveness. The results have indicated that the framework and its components are applicable and effective.