Preschool lessons: New research shows that teaching kids more and more, at ever-younger ages, may backfire. - Slate Magazine:
'via Blog this'
Great article on why preschools need to focus on play and exploration. I think that the same holds true for older children as well.
Thursday, September 19, 2013
Monday, September 16, 2013
New IES report of the effectiveness of Teach for America
I have to be honest and say that I have not read this report in its entirety yet. I have read the methods and results sections and that is what I will focus my critique on. I may a little biased against Teach for America (TFA) as I work in a College of Education who trains teachers in a traditional model. That being said, I am very passionate about teaching, I think that there are wonderful teachers out there doing a great job and a few that need to find another calling. I don't think that traditional teaching models have all the answers and I believe strongly that there is always room for improvement. So, if TFA teachers are doing a better job than teacher who get their credentials through a traditional teaching college, I want to know that and I want to understand why. Unfortunately, I don't believe that this study offers us any evidence that TFA teachers were in fact better as a group and they offer no information to help improve teacher preparation programs.
Link to Article
Research Design
I was initially put off by the authors repeatedly claiming that the fact that they used an "experimental design" and randomly assigned students to either a class taught by a TFA teacher (experimental group) or another teacher (control group) meant that there would be no initial differences between the two group of students. They state, "Random assignment was the key to the causal validity of these estimates because it ensured that students assigned to the TFA or Teachings Fellows teachers were no different, on average, than students assigned to comparison teachers in the same match at the time of random assignment." (pg. 21). This is simply untrue. The only thing random assignment guarantees is that any difference in the groups would be due to random chance rather than selection criteria. As a post-test only study there are a number of threats to internal validity (see http://www.socialresearchmethods.net/kb/intsoc.php for a short list of some of the problems). Many things can happen within any single class that have nothing to do with the teacher. In addition, they have lumped all other teaching programs into a single program. We have no idea where the teachers in the control group received their teaching credential and no idea how these different programs may have differed.
Analysis Methods and Results
I am not sure that I completely understand the regression analysis they did. Because they used a single regression analysis including all grade levels, they did a lot of statistical manipulations to create standardized measures for all the variables. All these adjustments make the interpretation of the final findings difficult to interpret. The final conclusion of the researchers was that students in classes being taught by TFA teachers on average scored 0.07 standard deviations (s.d.) higher that students in classes with non-TFA teachers, and that this difference was statistically significant. While they may have been able to find statistical significance after all that data manipulation, I am not convinced that this difference has any practical implications. They try to turn this small difference in s.d. scores into a 2.6 months of learning stating that this .07 standard deviations can be calculated to 26% of a years growth (pg.56). If I did the math right, that would indicate that students are only expected to grow roughly .27 standard deviations in a year. I really need to read the study they got that from.
Conclusions
What was not surprising was that there were some TFA teachers whose students did better than the students in the comparison class and there were some whose students did worse that the comparison teacher (pg.57). It would have been more informative if they had looked at both TFA and non-TFA prepared teachers who did better than their matched peer and looked for patterns that could explain why they did better. But since this study was all about TFA teachers, they did not do this.
Over all, I was very disappointed in the study. I felt like the researchers were trying to sell TFA rather than trying to explore how to better prepare teachers. I don't believe that this article offers any information for teacher preparation programs trying to improve their practices.
Link to Article
Research Design
I was initially put off by the authors repeatedly claiming that the fact that they used an "experimental design" and randomly assigned students to either a class taught by a TFA teacher (experimental group) or another teacher (control group) meant that there would be no initial differences between the two group of students. They state, "Random assignment was the key to the causal validity of these estimates because it ensured that students assigned to the TFA or Teachings Fellows teachers were no different, on average, than students assigned to comparison teachers in the same match at the time of random assignment." (pg. 21). This is simply untrue. The only thing random assignment guarantees is that any difference in the groups would be due to random chance rather than selection criteria. As a post-test only study there are a number of threats to internal validity (see http://www.socialresearchmethods.net/kb/intsoc.php for a short list of some of the problems). Many things can happen within any single class that have nothing to do with the teacher. In addition, they have lumped all other teaching programs into a single program. We have no idea where the teachers in the control group received their teaching credential and no idea how these different programs may have differed.
Analysis Methods and Results
I am not sure that I completely understand the regression analysis they did. Because they used a single regression analysis including all grade levels, they did a lot of statistical manipulations to create standardized measures for all the variables. All these adjustments make the interpretation of the final findings difficult to interpret. The final conclusion of the researchers was that students in classes being taught by TFA teachers on average scored 0.07 standard deviations (s.d.) higher that students in classes with non-TFA teachers, and that this difference was statistically significant. While they may have been able to find statistical significance after all that data manipulation, I am not convinced that this difference has any practical implications. They try to turn this small difference in s.d. scores into a 2.6 months of learning stating that this .07 standard deviations can be calculated to 26% of a years growth (pg.56). If I did the math right, that would indicate that students are only expected to grow roughly .27 standard deviations in a year. I really need to read the study they got that from.
Conclusions
What was not surprising was that there were some TFA teachers whose students did better than the students in the comparison class and there were some whose students did worse that the comparison teacher (pg.57). It would have been more informative if they had looked at both TFA and non-TFA prepared teachers who did better than their matched peer and looked for patterns that could explain why they did better. But since this study was all about TFA teachers, they did not do this.
Over all, I was very disappointed in the study. I felt like the researchers were trying to sell TFA rather than trying to explore how to better prepare teachers. I don't believe that this article offers any information for teacher preparation programs trying to improve their practices.
Thursday, September 5, 2013
The STEM Crisis Is a Myth - IEEE Spectrum
The STEM Crisis Is a Myth - IEEE Spectrum:
'via Blog this'
As an educator, an evaluator and a parent of school aged children this article gives me a lot to think about. As an evaluator, I am struck by the different definitions of STEM, the different calculations that are used to define whether there is a shortfall or a surplus in STEM workers, and how our biases are always present. Evaluators and researchers need to do their best not let their biases influence their results, but that it not always possible. That is one of the reasons why replication of studies is so important, so that we have results from a variety of sources and biases and can view data form different viewpoints to hopefully help us draw better conclusions. Politicians and business leaders have their own agendas and will only bring to light those studies that support those agendas. People have to look at the studies and the data themselves before making judgments and cannot blindly trust these leaders.
As a parent and educator, I agree with the author's conclusion that all students should be STEM literate, just as they should be literate in art, history and literature. The recent push for more STEM and less of the liberal arts classes worries me. I think the most successful people have a wide range of skills and knowledge. Understanding art, literature and history helps us understand how people think and behave, math and sciences help us understand how the world works. Why wouldn't we want all people to have all kinds of knowledge?
'via Blog this'
As an educator, an evaluator and a parent of school aged children this article gives me a lot to think about. As an evaluator, I am struck by the different definitions of STEM, the different calculations that are used to define whether there is a shortfall or a surplus in STEM workers, and how our biases are always present. Evaluators and researchers need to do their best not let their biases influence their results, but that it not always possible. That is one of the reasons why replication of studies is so important, so that we have results from a variety of sources and biases and can view data form different viewpoints to hopefully help us draw better conclusions. Politicians and business leaders have their own agendas and will only bring to light those studies that support those agendas. People have to look at the studies and the data themselves before making judgments and cannot blindly trust these leaders.
As a parent and educator, I agree with the author's conclusion that all students should be STEM literate, just as they should be literate in art, history and literature. The recent push for more STEM and less of the liberal arts classes worries me. I think the most successful people have a wide range of skills and knowledge. Understanding art, literature and history helps us understand how people think and behave, math and sciences help us understand how the world works. Why wouldn't we want all people to have all kinds of knowledge?
Friday, November 16, 2012
Planning Trips to Disneyland
What does a data geek do before a trip to Disneyland with the kids? Why survey the kids bout how much they like each ride and then plot the result of course! This was done about a year ago, the next time we go (which probably won't be another year or so) I am going to do this again and add the date as a field so we can see what kind of changes there were.
Saturday, November 10, 2012
More Data on CA budget
I found another data source, the California Legislative Analysis Office. The have a note on their site,
"Because of many changes over the years (including, but not limited to, changes in the sources of funding for certain state programs, deferrals of scheduled payments and tax collections, and other accounting changes), this data may not provide sufficient information to evaluate trends in general state spending, spending for particular programs, or state revenues. Consulting other sources, such as the LAO's annual Spending Plan publications, is advised when conducting such trend analyses."
I still thought it would be intersting to see what the data looks like graphed.
Monday, April 30, 2012
CA Budget Data View
I decided to create a data view of the history of the budget in California (CA) using the enacted CA Budget available from CA Department of Finance. While I think that this is interesting, I think that to get a better understanding of the budget, now I need to see how each category is spent.
Thursday, January 26, 2012
Less Large Scale Assessments in the Horizon?
I have been thinking about some the things that Governor Jerry Brown said about education in his State of the State address on January 18th (http://gov.ca.gov/news.php?id=17386). Below is the section of his speech related to education.
"Next, I want to say something about our schools. They consume more tax dollars than any other government activity and rightly so as they have a profound effect on our future. Since everyone goes to school, everyone thinks they know something about education and in a sense they do. But that doesn’t stop experts and academics and foundation consultants from offering their ideas — usually labeled reform and regularly changing at ten year intervals—on how to get kids learning more and better. It is salutary and even edifying that so much interest is shown in the next generation. Nevertheless, in a state with six million students, 300,000 teachers, deep economic divisions and a hundred different languages, some humility is called for.
"In that spirit, I offer these thoughts. First, responsibility must be clearly delineated between the various levels of power that have a stake in our educational system. What most needs to be avoided is concentrating more and more decision-making at the federal or state level. For better or worse, we depend on elected school boards and the principals and the teachers they hire. To me that means, we should set broad goals and have a good accountability system, leaving the real work to those closest to the students. Yes, we should demand continuous improvement in meeting our state standards but we should not impose excessive or detailed mandates.
"My budget proposes to replace categorical programs with a new weighted student formula that provides a basic level of funding with additional money for disadvantaged students and those struggling to learn English. This will give more authority to local school districts to fashion the kind of programs they see their students need. It will also create transparency, reduce bureaucracy and simplify complex funding streams.
"Given the cutbacks to education in recent years, it is imperative that California devote more tax dollars to this most basic of public services. If we are successful in passing the temporary taxes I have proposed and the economy continues to expand, schools will be in a much stronger position.
"No system, however, works without accountability. In California we have detailed state standards and lots of tests. Unfortunately, the resulting data is not provided until after the school year is over. Even today, the ranking of schools based on tests taken in April and May of 2011 is not available. I believe it is time to reduce the number of tests and get the results to teachers, principals and superintendents in weeks, not months. With timely data, principals and superintendents can better mentor and guide teachers as well as make sound evaluations of their performance. I also believe we need a qualitative system of assessments, such as a site visitation program where each classroom is visited, observed and evaluated. I will work with the State Board of Education to develop this proposal.
"The house of education is divided by powerful forces and strong emotions. My role as governor is not to choose sides but to listen, to engage and to lead. I will do that. I embrace both reform and tradition—not complacency. My hunch is that principals and teachers know the most, but I’ll take good ideas from wherever they come."
There is obviously a lot of rhetoric and it is difficult to determine exactly what action will take place, but there are some promising ideas in there. I liked his statements about the state testing and accountability. There is a lot of bureaucracy that goes along with "accountability" and I often wonder how much of the money in education is being spent on not just the tests and scoring of the tests (which is A LOT of money), but also on filling out required paperwork and reports that end up having no actual impact on students. As Governor Brown says, the state test results are not available for months after the students take the test. And the results do not provide teachers with any information that they do not already have. Teachers know which kids are below standard and which kids are above standard without these tests. What they need to know is how to support those kids who are below standard while still pushing the kids who are above standard. No test score is going to tell them that. It seems to me that "accountability" infers that teachers just are not trying hard enough with all the kids in their classroom. We want to make the teachers accountable for the kids who are not learning at school. I don't think that's fair. Kids come from so many different backgrounds and experiences that no one person can possibly reach all of them. That doesn't mean we don't try, but it means that we offer more support for kids and teachers, not just add more levels of bureaucracy that end up costing more money and don't do anything to support our kids.
"Next, I want to say something about our schools. They consume more tax dollars than any other government activity and rightly so as they have a profound effect on our future. Since everyone goes to school, everyone thinks they know something about education and in a sense they do. But that doesn’t stop experts and academics and foundation consultants from offering their ideas — usually labeled reform and regularly changing at ten year intervals—on how to get kids learning more and better. It is salutary and even edifying that so much interest is shown in the next generation. Nevertheless, in a state with six million students, 300,000 teachers, deep economic divisions and a hundred different languages, some humility is called for.
"In that spirit, I offer these thoughts. First, responsibility must be clearly delineated between the various levels of power that have a stake in our educational system. What most needs to be avoided is concentrating more and more decision-making at the federal or state level. For better or worse, we depend on elected school boards and the principals and the teachers they hire. To me that means, we should set broad goals and have a good accountability system, leaving the real work to those closest to the students. Yes, we should demand continuous improvement in meeting our state standards but we should not impose excessive or detailed mandates.
"My budget proposes to replace categorical programs with a new weighted student formula that provides a basic level of funding with additional money for disadvantaged students and those struggling to learn English. This will give more authority to local school districts to fashion the kind of programs they see their students need. It will also create transparency, reduce bureaucracy and simplify complex funding streams.
"Given the cutbacks to education in recent years, it is imperative that California devote more tax dollars to this most basic of public services. If we are successful in passing the temporary taxes I have proposed and the economy continues to expand, schools will be in a much stronger position.
"No system, however, works without accountability. In California we have detailed state standards and lots of tests. Unfortunately, the resulting data is not provided until after the school year is over. Even today, the ranking of schools based on tests taken in April and May of 2011 is not available. I believe it is time to reduce the number of tests and get the results to teachers, principals and superintendents in weeks, not months. With timely data, principals and superintendents can better mentor and guide teachers as well as make sound evaluations of their performance. I also believe we need a qualitative system of assessments, such as a site visitation program where each classroom is visited, observed and evaluated. I will work with the State Board of Education to develop this proposal.
"The house of education is divided by powerful forces and strong emotions. My role as governor is not to choose sides but to listen, to engage and to lead. I will do that. I embrace both reform and tradition—not complacency. My hunch is that principals and teachers know the most, but I’ll take good ideas from wherever they come."
There is obviously a lot of rhetoric and it is difficult to determine exactly what action will take place, but there are some promising ideas in there. I liked his statements about the state testing and accountability. There is a lot of bureaucracy that goes along with "accountability" and I often wonder how much of the money in education is being spent on not just the tests and scoring of the tests (which is A LOT of money), but also on filling out required paperwork and reports that end up having no actual impact on students. As Governor Brown says, the state test results are not available for months after the students take the test. And the results do not provide teachers with any information that they do not already have. Teachers know which kids are below standard and which kids are above standard without these tests. What they need to know is how to support those kids who are below standard while still pushing the kids who are above standard. No test score is going to tell them that. It seems to me that "accountability" infers that teachers just are not trying hard enough with all the kids in their classroom. We want to make the teachers accountable for the kids who are not learning at school. I don't think that's fair. Kids come from so many different backgrounds and experiences that no one person can possibly reach all of them. That doesn't mean we don't try, but it means that we offer more support for kids and teachers, not just add more levels of bureaucracy that end up costing more money and don't do anything to support our kids.
Subscribe to:
Posts (Atom)