The Development of the STEM (Science, Technology, Engineering, and Mathematics) Attitude and Motivation Survey Towards Secondary School Students

The purpose of the research was to determine the secondary school students’ attitude and motivation toward STEM. For this purpose, two field studies for developing a survey were conducted. The first field study, pilot study, was performed for collecting the research data and making statistical analysis of the research data. After completing statistical analysis procedures (Explanatory Factor Analysis and Confirmatory Factor Analysis) of the pilot survey, the reliability and validity of the pilot survey was calculated. The statistical results indicated that the pilot survey was reliable and it also had four sub-factors. The second field study, main study, was performed for comparing and discussing the research data. The research was applied to 2912 students at different 14 secondary schools during the academic year of 2018-2019 at Izmir, Turkey. The students’ results in the metropolitan and suburban regions of the city were compared. The results indicated the decrease in the attitude and motivation of students at higher levels in both regions toward STEM. When the results of the research were evaluated in terms of the sub-factors, the science and engineering values of secondary school students in the suburban were higher than the values of the students in metropolitan. It was not found a significant difference between the students’ technology and mathematics values for both metropolitan and suburban. There was a significant difference between the 7th grade students’ science, technology, engineering, and mathematics values instructed in metropolitan and suburban in favor of the students instructed in suburban. It was calculated that there was not a significant difference between the 6th and 8th grade students’ all values for both the regions. It was only calculated that there was a significant difference between the 5th grade students’ mathematics value. Investigations on gender factors showed that the female students’ science, technology, engineering, and mathematics values were generally lower than the male students’ values for both the regions.

Introduction each discipline is able to be thought of as a cluster. Educators and researchers can focus on combining four disciplines and bridging the relationships among four disciplines in today's technological world (Sanders, 2009;So et al., 2018).
The main purpose of STEM education is to foster qualified and skilled manpower (Luo et al., 2019). Also the purposes of STEM education are to develop industry and technology, to raise welfare levels, to increase literacy rates, etc. (Christensen, Knezek and Tyler-Wood, 2015;Hanson and Krywult-Albanska, 2020;Unfried et al., 2015). Bybee (2011, p.6) determined that integrated STEM education focuses on eight factors which are "asking questions and defining problems, developing and using models, planning and carrying out investigations, analyzing and interpreting data, using mathematics and computational thinking, constructing explanations, designing solutions, engaging in argument from evidence, obtaining, evaluating, and communicating information". Students may begin to learn by becoming aware of the eight fundamental factors. Students may understand how to improve the design, inquiry, and analysis process and how to combine innovation and technology in a true STEM education environment. Students are able to study on projects and laboratory investigations with the STEM curriculum in cooperative learning groups. Besides, teachers may explain to students what they are to do and how they are taught in these education settings in order to develop deeper and richer understandings and abilities (Hasanah, 2020;Li et al., 2020;Margot and Kettler, 2019). STEM helps students improve 21 st century skills covered flexibility, adaptability, creativity, information resources management, media and technology literacy, self-management, social skills etc. and specifically ensures that students are sensitive citizens about energy efficiency, air and environmental pollution, climate change, use of natural sources, national security, personal healthy, economics, politics, and cultural values, personal, social and global issues, etc. STEM also makes significant contributions to the importance of science, the emphasis of technology, and the recognition of engineering in all school programs (Bybee, 2010b;Sanders, 2009).
STEM will play an important role in students' career choices nowadays and in the near future. Many students will generally focus on STEM-based jobs instead of natural sciences, medical fields, engineering jobs. Employers in industries will also consider qualified workers with STEM-related skills for filling empty and critical positions (Abe and Chikoko, 2020;Brown et al., 2011). Therefore determining and guiding students' academic choices are quite important at the primary and/or secondary school levels in today's world. The academic choices of students have lately been begun to investigate in depth by researchers (Craig et al., 2018;Maltese and Tai, 2011;Sadler et al., 2012;Unfried et al., 2015).
Integrated STEM education plays an important role in improving students' problem solving skills, creative thinking, social skills, communication and collaboration skills, self-management skills, critical thinking, and analytical thinking (Brown et al., 2011). When integrated STEM education based on constructivist (Bransford, Brown and Cocking, 2000;Sanders, 2009) will be applied at schools or universities, it will pave the way for some radical changes according to the present education systems such as teaching methods (inquiry-based learning, project-based learning, problem-based learning, design-based learning, etc.) and assessment of materials, teacher training, class environment, etc. However, these kinds of radical changes are also responsible fortaken responsibility time, effort, support and financial commitment (Leung, 2020;Merrill, 2001;Sanders, 2009). Integrated STEM education contributes rediscovering present scientific knowledge based on learning science and mathematics, developing engineering and technological design processes, thinking about real world issues, enhancing analytical thinking, seeking solutions toward social, economic, and environmental issues, and contributing to the establishment of interdisciplinary relations of students at all grade levels.
The most important feature that distinguishes this study from other studies is the determination of secondary school students' tendencies towards science, technology, engineering and mathematics. After the STEM trends of the secondary school students are determined, program development experts will need to create a curriculum according to the determined trends. Teachers will need to determine alternative teaching strategies in order to achieve the goals determined according to the curriculum prepared. School administrations will need to provide the necessary equipment and materials to enable teachers and students to carry out activities based on research, project, collaboration, etc. The study is mainly focused on the tendency of the secondary school students. The other results of the research are focused on attracting the attention of school administrators, curriculum development experts and teachers. . The development of the STEM (science, technology, engineering, and mathematics) attitude and motivation survey towards secondary school students, International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE), 105-119.

STEM Education in Turkey
In Turkey, science and mathematics classes are taught to the students at the secondary school level by using traditional educational approaches rather than integrated STEM education which are expressed together with science, technology, engineering and mathematics (Cevik, 2017;Yamak, Bulut and Dundar, 2014). In this approach many science teachers just teach the students the area course without thinking about different disciplines, and they usually do not mention science applications toward engineering and technology at their own classes. The students have science, mathematics, computers, etc. classes but these classes are separately taught to the students by using traditional education which is focused on one discipline (e.g., science, mathematics). Therefore they may not synthesize instructed subjects, they might easily forget independent of taught knowledge, they may not be interested in STEM subjects, and they might not choose a STEM career, they can graduate from secondary and/or high schools with lower science and/or mathematics. Besides, the students are not taught engineering and technology classes in their own school curriculum and the national educational programs.
The primary goal of the present study is defining the attitude and motivation of the secondary school students by thinking Turkish students' demographic characteristics with a developed STEM attitude and motivation survey with the support of the Ministry of National Education. It is expected that this developed STEM survey in the research will fill the gap in the literature for thinking countries like us.

Materials and Methods
The present research covered two sections. The first section examined the pilot survey development procedures. The section consisted of conducting literature review, creating a large pool of the pilot survey items, taking expert opinions for items of the pilot survey, and performing factor analysis based on "exploratory factor analysis" and "confirmatory factor analysis". The second section contained the application of the survey developed for attitude and motivation toward STEM. This section discussed the findings of the developed survey.

The Pilot Survey Development Procedures
The literature on STEM surveys (Faber et al., 2013;Kier et al., 2014;Tyler-Wood, Knezek and Christensen, 2010) measured the attitude and motivation of the students toward STEM disciplines was reviewed by using of ERIC, Google Scholar, and Science Direct. Volunteer 30 students were asked to write their own views on the integrated STEM education item by item. The similar expressions were considered based on students' perspectives. Examined surveys and written views helped to determine items of the pilot survey and made a significant contribution of the pilot survey with 69 items. The pilot survey was sent to the five STEM specialists and six science, mathematics, and computer teachers working in STEM education for assessing the content validity. According to the expert recommendations, the pilot survey with 69 items on the five-point Likert scale was reviewed for each item. The items were scaled from strongly agree "5" to strongly disagree "1".

Participants
The data of the pilot survey were collected from 1335 students in seven public secondary schools during the academic year of 2018-2019 in the third largest city (İzmir) of Turkey. The schools had not ever implemented integrated STEM education program. The pilot study was performed in two regions of Izmir in order to generalize the obtained findings of the research. The first region was metropolitan. The region covering two districts were randomly chosen by thinking of time and budget of the research. The second region was suburban. The region was close to the city center and was the author's workplace. All of the students were voluntary in the research. The students were anonymously asked to fill the survey and the surveys were coded randomly.
The pilot survey of 192 students who filled the survey incorrectly and/or incompletely were not evaluated. Therefore the statistical analysis of the pilot survey was conducted on 1143 students. Six hundred and thirty-four of 1143 students were female and the rest of them were male. Table 1 indicates the demographic variables of the students. The 5 th grade students were not taken part in the research. Because they went through the adaptation period for transition from primary to secondary school.

Table 1 Demographic variables of the students
Students were approximately given ten minutes to fill out the pilot survey in their own class time. Before handing out the pilot survey, the explanatory information about the pilot survey were briefly presented to the students.

Data Analysis
The statistical analysis of the pilot survey was examined in two subtitles which are Explanatory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA). EFA was performed by SPSS Statistics 25 while CFA was conducted on AMOS 25 Graphics program.

Explanatory Factor Analysis (EFA) of the Pilot Survey
The pilot survey according to EFA consisted of 34 items. All items in the pilot survey were positive statements. Some statistical analysis data concerning the pilot survey were given as follows: Bartlett's test of sphericity was measured as 20429.593, df=561, p<.001, KMO (Kaiser-Meyer-Olkin) value was calculated as .91, total explained variance value was measured as 57.94% and Cronbach's Alpha value was calculated as .84. The obtained statistical data showed that the factor analysis for the pilot survey was appropriate (Hair et al., 2014). The pilot survey found four sub-factors. The sub-factors were named considering the content of the research. The first factor consisted of 7 items and was called "Science" (S). The second factor comprised 8 items and was called "Technology" (T). The third factor consisted of 10 items and was called "Engineering" (E). The last factor comprised 9 items and was called "Mathematics" (M). Figure 1 indicates the relationship between the items and eigenvalues of the pilot survey. The factor loadings of the items (see in Appendix 1) are over .65 and the eigenvalues of sub-factors were above "1". When the examined studies (Hair et al., 2014;Tabachnick and Fidell, 2012) in the literature were evaluated, it could be said that the values obtained from explanatory factor analysis are "high" and "acceptable".  . The development of the STEM (science, technology, engineering, and mathematics) attitude and motivation survey towards secondary school students, International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE), 105-119.

Confirmatory Factor Analysis (CFA) of the Pilot Survey
CFA was conducted to support EFA results of the pilot survey. CFA' values are expected results between "acceptable fit" and "good fit". After the statistical analyses were performed, the modification index values were observed between "acceptable fit" and "good fit". Table 2 illustrates the CFA measurement values.  (Karagoz, 2016;Schermelleh-Engel, Moosbrugger, and Müller, 2003) The results obtained from CFA according to examined studies in the literature (Byrne, 2013;Hu and Bentler, 1999;Karagoz, 2016;Schermelleh-Engel, Moosbrugger and Müller, 2003) indicated that the calculated measurement values are "valid", and "reliable". Figure 2 shows the factor loading values of the pilot survey according to CFA. . The development of the STEM (science, technology, engineering, and mathematics) attitude and motivation survey towards secondary school students, International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE),9(1), 105-119.

Results
The Application of the Developed Survey A new field study was conducted to evaluate the results of the developed survey. The second field study was performed by different students except the schools visited for the pilot survey. In this study, the following research questions were investigated: 1) Is there a significant difference between the grade levels of the students toward the STEM attitude and motivation surveys? 2) Is there a significant difference between the female students instructed in metropolitan and suburban toward STEM attitude . The development of the STEM (science, technology, engineering, and mathematics) attitude and motivation survey towards secondary school students, International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE), 105-119. and motivation surveys? 3) Is there a significant difference between the male students instructed in metropolitan and suburban toward STEM attitude and motivation surveys?
The data of the developed survey were collected from 1577 students in seven different secondary schools during the academic year of 2018-2019 in the third largest city (İzmir) of Turkey. Three hundred and forty-seven of 1577 student surveys were not evaluated because of missing data. One thousand two hundred and thirty students filled out the survey in full. Six hundred and eighty-three students live in the metropolitan and the others live in the suburbs. Six hundred and fourteen of 1230 students were female and the other students were male. Table 3 indicates the demographic characteristics distributions according to region, gender and grade level of the students. In this process for the developed survey, 5 th grade students' data were used to compare and discuss the results of the students at all grade levels.

Table 3
The demographic characteristics of the students Note: M: Metropolitan; S: Suburban Table 4 demonstrates the independent samples t-test results according to the students' grade levels. It was realized that there was not a significant difference between the 5 th grade students' science, technology, engineering and total STEM values except their own mathematics value. Also there was not a significant difference between the 6 th grade students' and between the 8 th grade students' science, technology, engineering, mathematics and total STEM values. It was found that there was a significant difference between the 7 th grade students' all values.

Table 4
The comparative results of the survey according to the grade levels Table 5 presents the descriptive statistics of students' attitude and motivation towards STEM in metropolitan and suburban areas. Table 6 and Table 7 also show the mean of students' attitude and motivation toward STEM according to the grade level, gender and region. The answers of the students studied in metropolitan were in the range of 34 and 170 points while the answers of the students studied in sub-urban were in the range of 45 and 170 points. The results of the students instructed in metropolitan and suburban were compared with independent samples t-test. The statistical analysis according to the sub-factors of the developed survey were calculated as t (1228) =3.18 p<.05 for "Science", t (1228) =1.63 p>.05 for "Technology", t (1228) =2.11 p<.05 for "Engineering", t (1228) =.31 p>.05 for "Mathematics", and finally t (1228) =2.28 p<.05 for "STEM" respectively. The results of the female and male students instructed in metropolitan and suburban were compared with independent samples t-test. The statistical analysis results of the female students according to the sub-factors of the developed survey were calculated as t (612) =1.61 p>.05 for "Science", t (612) =1.29 p>.05 for "Technology", t (612) =1.27 p>.05 for "Engineering", t (612) =.43 p>.05 for "Mathematics", and finally t (612) =1.44 p>.05 for "STEM" respectively. The statistical analysis results of the male students according to the subfactors of the developed survey were calculated as t (614) =2.81 p<.05 for "Science", t (614) =1.23 p>.05 for "Technology", t (614) =2.03 p<.05 for "Engineering", t (614) =.06 p>.05 for "Mathematics", and finally t (614) =1.90 p>.05 for "STEM" respectively.
After the descriptive statistics as shown in Table 6 and Table 7 were determined, the comparative analysis was calculated according to the grade levels and gender of the students in metropolitan and suburban areas. It was calculated that there was not a significant difference between the male students' science, technology, engineering, mathematics and total STEM values and between the female students' science, technology and total STEM values for 5 th -8 th grade levels.

Table 6
The means of the students' survey according to the grade level and gender in metropolitan Table 7 The means of the students' survey according to the grade level and gender in suburban

Discussions
In this section, the results obtained from the statistical analysis were generally discussed. Figure 3 presents that the attitude and motivation of the students instructed in metropolitan and suburban towards STEM broadly decreased from 5 th grade level to 8 th grade level. The attitude and motivation of the students studying in the metropolitan area diminished by 9 percent while the attitude and motivation of the students studying in the suburbs decreased by 6 percent.

Figure 3. The students' STEM mean value according to the grade level and region
According to the first research question, there was not a meaningful difference between the students' values instructed in the 5 th grade level, the 6 th grade level and 8 th grade level except the 7 th grade level toward STEM attitude and motivation survey.
The 5 th grade and the 6 th grade students in metropolitan and suburban might have spent more time for the orientation period from primary school to the secondary school. The students in the 8 th grade level were prepared for high school exams in order to enroll in state, private, science, and vocational high schools. These students might not have adequately shown the interest in STEM disciplines because of exam stress and anxiety. The students in the 7 th grade level might have been more interested in STEM disciplines because they quickly completed the orientation period and they did not have any high school exam stress and anxiety. The 7 th grade students in the suburbs were more curious and enthusiastic toward STEM disciplines than the students in the metropolitan. They followed the developments and innovations in science, technology and engineering fields. Shortly, the students in suburban areas focused on less cultural activities, more on STEM disciplines.
The decreases in the graph show that the STEM viewpoints of the students in metropolitan and suburban negatively influenced while going from the 5 th grade level to the 8 th grade level. The decline on the attitude and motivation of the students toward STEM could be explained in terms of the students, the teachers and the schools as follows: a) Many students may not be interested in STEM, they may not want to study science and mathematics classes, they may have been guided to different disciplines (e.g., political sciences, visual art, social science, sports science etc.) in their career choice except STEM disciplines. b) The teachers may not have received any education on STEM, so the students may not have been adequately informed and guided about STEM, they may have used traditional educational methods and approaches instead of integrated STEM education based on project-based learning, problem-based learning, inquiry-based learning, cooperative learning etc. Science teachers generally performed in their own lessons in the classroom environment instead of the laboratory environment. Therefore many students may have decreased interest and curiosity towards STEM. c) The necessary and sufficient physical conditions (e.g., budget, laboratory tools, class, computers, internet, etc.) of the schools may not be convenient for integrated STEM education in terms of the secondary schools. Figure 4a and Figure 4b indicate the total STEM value of the female and male students instructed in metropolitan and suburban. . The development of the STEM (science, technology, engineering, and mathematics) attitude and motivation survey towards secondary school students, International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE),9(1), 105-119.
www.ijcrsee.com 114 Figure 4a. Female students' STEM mean value Figure 4b. Male students' STEM mean value according to the grade level and region according to the grade level and region The STEM value of the female students instructed in metropolitan and suburban decreased from the 5 th grade level to the 8 th grade level. The decline values were calculated as 16 percent for metropolitan and 6 percent for suburban. According to the second research question, there was not a meaningful difference between the female students' values instructed in metropolitan and suburban toward STEM attitude and motivation surveys. But, total STEM values of the female students in the metropolitan generally decreased from the 5 th grade level to the 8 th grade level. Total STEM values of the female students in suburban areas varied from the 5 th grade level to the 8 th grade level. The STEM attitude and motivation values of the female students decreased at the 6 th grade level. The values of the female students increased at the 7 th grade level and negatively showed small change in the 8 th grade level. The female students in suburban areas were more interested in science, technology, engineering, and mathematics disciplines according to the female students instructed in metropolitan areas.
The STEM value of the male students instructed in metropolitan and suburban also decreased from the 5 th grade level to the 8 th grade level. The decline values were found as 7 percent for metropolitan and 5 percent for suburban respectively. According to the third research question, there was not a meaningful difference between the male students instructed in metropolitan and suburban toward STEM attitude and motivation surveys. But, total STEM values of the male students in metropolitan generally decreased from the 5 th grade level to the 8 th grade level. Total STEM values of male students instructed in metropolitan schools began to rise from the 7 th grade level to the 8 th grade level. Total STEM values of the male students increased from the 5 th grade level to the 7 th grade level and then the values of the male students decreased at the 8 th grade level. It could be said that the male students as in the female students in suburban might have been more willing and interested in science, technology, engineering, and mathematics disciplines according to the male students instructed in metropolitan.
Broadly, the total STEM values of the male and female students instructed in metropolitan were lower than the total STEM values of the male and female students instructed in suburban. The attitude and motivation values toward STEM of the female students in metropolitan according to grade levels were lower than the STEM values of the male students. The STEM value of female students instructed in metropolitan areas declined from 131 points to 105 points. It could be said that as the grade levels of the students in suburban and metropolitan increases, the students' attitude and motivation values toward STEM decreases. These decline values might be attributed to following points: The female and male students in metropolitan may not be interested in STEM, they might have gotten around to cultural ac-tivities more time instead of learning science and mathematics classes, they might have difficulties in learning science and mathematics. Figure 5a and Figure 6a indicate the science mean values of the female and male students in metropolitan and suburban areas. Science values of the male students in metropolitan and suburban changed at the same rate (6%). Science values of the female students in suburban areas did not considerably change. But, the science value of the female students in metropolitan areas considerably decreased from the 5 th grade level to the 8 th grade level. The decrease was approximately 17 percent. These results indicated that many of the female students may have problems visualizing abstract concepts and understanding fundamental concepts and principles about science. The science values of the female students may also have been affected from the following factors: a) the cultural level of the society may be an important factor in the choice of profession of the female students. b) the female students might worry about the future, c) psychological and sociological support may not be provided to the female students during their career choices, d) the female students may see themselves insufficient for science . The development of the STEM (science, technology, engineering, and mathematics) attitude and motivation survey towards secondary school students, International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE),[105][106][107][108][109][110][111][112][113][114][115][116][117][118][119] and mathematics classes, e) the female students might not show any interest in science and mathematics classes. Figure 5b and Figure 6b present the technology mean values of the female and male students in metropolitan and suburban. The technology value of the female students dramatically decreased from the 5 th grade level to the 8 th grade level. This decline was roughly 15 percent. The technology values of the other students in metropolitan and suburban areas negatively changed at the same rate. The rate of change was about 8 percent. Concern with technology of the female students instructed in metropolitan was lower than other students. As understood from the graphics, the female students were not interested in technology, the female students may have directed their interests and curiosity to different disciplines. Besides, the students may not generally have technology design class and computer class in their schools, they might not have enough computers and internet in their school, they may not have technology and design teachers in their schools. Figure 5c and Figure 6c illustrate the engineering mean values of the female and male students in metropolitan and suburban. The engineering values of the male students in metropolitan and the female students in suburban did not change. The engineering value of the male students in suburban schools slightly changed from the 5 th grade level to the 8 th grade level with a 2 percent decline. The engineering value of the female student declined from the 5 th grade level to the 8 th grade level by 12 percent. The engineering values presented show that many students except the female students in metropolitan areas may take interest in engineering. Many students want to pursue their own careers on in engineering education. Most students may enjoy repairing electronic and mechanical devices. They like to see engineering applications in their daily lives. Many students also prefer the engineering profession to lead a more comfortable life and earn a lot of money. Figure 5d and Figure 6d present the mathematics mean values of the female and male students in metropolitan and suburban. The mathematics values of all students decreased. The decline of mathematics values of the female (18%) and male (13%) students trained in metropolitan was higher than the decrease of mathematics values of the female (11%) and male (5%) students trained in suburban. The findings indicated that many students do not want to choose a career in this field. Many students might not like to learn mathematics, they might have difficulty performing and understanding mathematical operations, teachers may not have experience in subject knowledge and pedagogical knowledge on the students' attitudes.

Conclusions
The difference of this study from other conducted studies that are adaptation of the STEM questionnaire or that focuses on the development and approval procedures of a new STEM questionnaire (Barlow et al., 2020;Faber et al., 2013;Guzey, Harwell and Moore, 2014;Paul, Maltese and Valdivia, 2020;Unfried et al., 2015), focusing on developing the "STEM attitude and motivation survey" in secondary school and discussing the collected data in terms of grade and region with the help of second field research. Statistical analysis results of the attitude and motivation survey toward STEM indicated that the reliability and validity of the developed survey was reliable and helpful to identify the students' attitude and motivation for science, technology, engineering, and mathematics fields. The research was conducted to the secondary school students instructed in metropolitan and suburban. The results of the developed survey were presented that the total values of the students instructed in suburban were higher than the total values of the students instructed in metropolitan. The attitude and motivation of both the female students and the male students in suburban and metropolitan toward STEM decreased from the 5 th grade level to 8 th grade level. This decline in suburban students was relatively less according to metropolitan students. The science, technology, and engineering values of the female stu-dents instructed in metropolitan were comparatively lower than the male students' values. The similar studies (Blatt et al., 2020;Kurban and Cabrera, 2020;Whitcomb et al., 2020) in the STEM field support the results of the research. The matematics values of the female and male students remained at the same level from the 5 th grade level to the 8 th grade level and the mathematics values of the students declined as in other sub-factors. The science and mathematics values of the female students instructed in suburban schools were slightly higher than the male students' values. The technology and engineering values of the male students in the suburbs was higher than the female students' values. Especially the attitude and motivation for engineering field of the male students were higher than the other fields.

Recommendations
Some recommendations based on the results of the research could be listed as follows: Integrated STEM education might be applied at all grade levels in the course of time. A sustainable education policy should be followed to discuss the results of the integrated STEM education. Field teachers might be . The development of the STEM (science, technology, engineering, and mathematics) attitude and motivation survey towards secondary school students, International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE),[105][106][107][108][109][110][111][112][113][114][115][116][117][118][119] given training about integrated STEM education. The importance of integrated STEM education may be explained to policy makers and administrators. The importance of integrated STEM education might be explained to the society. The importance of technology and engineering could be explained to the students. Field teachers may encourage students to do hands-on activities in a laboratory environment. Science, technology design and mathematics teachers in the schools might work in cooperation by using integrated teaching and learning approaches. School administrators may establish a strong interaction and communication between school and faculty of engineering and natural sciences for the development of the students. The teachers might integrate their own classes with technology and engineering, explain relationships between technology and engineering with the help of richer and deeper integrated STEM curriculum programs. Researchers could examine the effects of integrated STEM education on students' academic achievement with the help of the experimental studies. There are not enough studies about students' achievement in the literature. More studies and data in order to evaluate and interpret the effects of the integrated STEM education are needed in the literature.

Limitations
The limitations of the research are given as follows: The pilot survey and the developed survey can be applied to many more secondary school students in different cities of Turkey. The pilot survey was not applied to the 5 th grade students. Because they experienced the orientation process. The relationships between academic achievement in science and mathematics and attitude and motivation towards STEM surveys of the secondary school students can be examined. The effects of parents' education level on students' perspective on STEM fields should be investigated.