Evaluating Research
My recent posts have included a research plan I designed and may be interested in some day implementing. While I am not sure that I will be actively researching any time in the near future, I can be sure that it will be necessary for me to analyze and evaluate educational research and decide on the impact, if any, it will have on my practices. As a more cognizant consumer of educational research I can offer the following as advice for others looking for high quality research. 1. Look for who is conducting, publishing, and funding the research. If there is any indication of conflicts of interest or a sense of unprofessionalism, it is cause to question the research. 2. Be sure that the description indicates not just what was done but why it is important and why it was done the way it was. 3. The research should also clearly describe the conclusions and implications, and not confuse correlation with cause. 4. Be ware of the term scientific research, or overarching claims about what research shows. 5. The research should refer to other literature relevant to the topic, and should provide a variety of sources both in support and in opposition to the hypothesis.
Information summarized from CEP822 and Brian Carpenter’s podcast.
Research Plan
Research Question:
What is the effect of student creation of digital stories on their understanding of history concepts in an elementary setting?
Sample:
The sample studied will include two fourth grade classes (I will refer to them as Class A and Class B), 50 total students. The students are 9 and 10 years old. They are a racially and academically diverse group, and there are 25 male students and 25 female students. The classes are located in Okemos, Michigan, a suburban and relatively affluent community. More details will follow, but each group will be taught social studies by the same teacher for two consecutive units.
This sample is useful for this study for several reasons. First is the possibility for the same teacher to instruct both classes so there is not a question about difference in instruction in the results. Also, the diversity found in the classrooms both in ethnicity and academic ability creates an interesting cross section. Although generally the community has high socio economic status, there is also a range of family income levels. The age level is appropriate both for the use of the technology, and when considering using narrative for teaching social studies.
Study Design
For the first unit, The Revolutionary War, Class A will participate in digital stories, and Class B will not. For the second unit, The Civil War, Class B will participate in digital stories and Class A will not. (The unit with the digital stories for each unit will be labeled with an asterisk from this point on.) These units are not consecutive but were chosen because of the similarities in the nature of the content.
Each unit will be taught in a similar fashion with lecture, readings, videos, etc. and a focus on narrative. The difference will be that for one unit each class will be asked to create a digital story based on the content of the unit. Each student will create his or her own story focused on one aspect of the unit. The two classes will be compared against each other for the two separate unit to discern the effects that creating a digital story has on the students’ understanding of the social studies content. This will allow for both classes to experience creating digital stories and give two separate opportunities to observe and analyze differences.
Data Sources
The data for this research study will come from four separate sources, two assessments, and two interviews. The interviews will take place before and after each class’ digital story unit, and the results of each set of interviews will be compared. (Class A: One pre and one post interview for Unit 1. Class B: One pre and one post interview for Unit 2). The assessments will take place at the end of each of the two units for each class. Class A: Test for Unit 1*, Test for Unit 2. Class B: Test for Unit 1. Test for Unit
2*).
Pre-Interview
Before each class begins the unit with digital stories, they will be interviewed to gauge their understanding of the content and their feelings about the previous unit. This pre-interview will include a variety of questions to see where students are before the unit begins.
There will be content specific questions:
What do you know about the Civil War?
Why was the Civil War Important to American History?
What important historical figures can you name from the Civil War?
Also, there will be questions about attitude.
How do you feel about Social Studies?
What is easy about social studies?
What is difficult about social studies?
Assessment
Each class will take a test at the end of each unit. These tests will be the typical tests given at the end of each social studies unit. The tests include multiple choice, matching, and essay questions. The multiple choice and matching questions will cover vocabulary words and other details of the units. The essay questions will be considered more for the purpose of the study and will necessitate more substantial answers.
Essay questions will include:
Describe some of the factors that led to the Civil War?
Describe some of the factors that led to the Revolutionary War?
Post-Interview
After each group has participated and been tested, they will each be re-interviewed. This interview will allow me to see how their understandings and attitudes have changed over the course of the unit. An interview allows for a conversation and students to express more easily (in narrative if needed) what they are thinking, and will generate more complete answers than a survey. The questions will be the same as those listed for the pre-interview, with a few additional questions.
Describe your experience with the test, was it harder or easier than usual? Why?
What is the most important thing you have learned from this unit?
What did you learn from creating a digital story?
Procedure
Class A
Interview students individually before the unit begins.
Begin Unit 1*: The Revolutionary War
Teach Unit
Students work on digital story assignment
Give Assessment for Unit 1*
Interview students after unit individually.
Teach Unit 2: The Civil War
Give Unit 2 Assessment
Class B
Teach Unit 1: The Revolutionary Way
Give Unit 1 Assessment
Interview students before the unit begins
Begin Unit 2*: The Civil War
Teach Unit
Students work on digital story assignment.
Give Students Unit 2 Assessment.
Interview students after unit
This procedure allows for both classes to learn one unit with using the digital stories. It also allows for a variety of information to be collected.
Data Analysis
In order to gauge the effect of the digital stories on student understanding of history content, the following four data comparisons will be made:
1. Class A, Unit 1* Test to Class A, Unit 2 Test
2. Class B, Unit 1 Test to Class B, Unit 2* Text
3. Class A, Unit 1* Pre and Post Interviews
4. Class B, Unit 2* Pre and Post Interviews
Interviews
Notes from interviews will be compared to see how students understanding of content and attitudes changed. Specifically, I will look for students feelings towards social studies and what is difficult and easy, and see if the use of digital narrative changed student opinions.
Post-Assessment
As mentioned above, students will take the standard exam given at the end of each unit, which includes multiple choice, matching, and essay questions. I am interested in the grades on the multiple choice and essay questions and will compare those scores for each group. More careful attention will be paid to student’s answers to the essay questions, which give students a chance to express what they have learned. Using essay rubrics I will evaluate students answers for the details they include and the connections they make between content.
Ten Things to Think
Why “Ten Things to Think”?
Sometimes it is difficult for
teachers interested in using technology in the classroom to know where to start. There are endless tools, software, hardware, and people who will insist they have the solution, the right thing for your students, the technology everyone needs to know how to use. While I have my favorite tools and software I could tell you all about, I think it is more important to start out with ways of thinking. Learning the ins and outs of tools and hardware will be daunting and meaningless without first establishing how you think about using technology. You need to find what is right for you, and these general things to think will give you a framework and a starting point. Then, the possibilities are endless to have a great experience with technology in your classroom. I hope they are helpful to you in your technology adventures.
There is a main document which can be downloaded here. Also if you visit http://msu.edu/~kelleral/T4.html there is more information and supporting documents including a poster and a lesson checklist available. All of these things are in an early draft, I have planned edits that will be up this summer. If you have any suggestions or comments, please leave them here or email me (kelleral@msu.edu).
