Online testing, are we setting our cadets up for failure?

[manfredvonrichthofen]

Really, are we setting our cadets up for failure with online testing?

From what I can tell, it sets cadets up for future failure in their higher ranks. Allowing tests for advancement to be open book makes it to where cadets can't make on the spot decisions as they need to be able to do. Part of the purpose of testing (a large part) is to ensure that the knowledge needed is sinking in. I could understand that there may be a need in some areas where meetings may not be able to be attended regularly, but in most squadrons this is a non-issue. Most of the time not showing up to a meeting is personal choice or because of a prior commitment. I just think that online open book testing is going to make achievements that were coveted such as the Mitchell and Spaatz easier to attain, and ultimately mean less. I do want to see every cadet achieve these awards, but not at the expense of the meaning of the awards. Every cadet has had the ability to achieve these awards, but many don't because they are so hard to get, they are time consuming and require commitment that some cadets just don't have or don't want to give.

By making testing available online it promotes the idea that no one needs to really know anything other than how to look something up. Leadership testing ensures that cadets understand what they need to and can retain the info that they need in order to effectively lead. If we leave it up to them to take the test online with an open book, then they will be ill equipped to lead when their time comes.

Do you advocate testing online to your cadets?

[Eclipse]

Yes, because that is "the Way" and this is 2011, not to mention that these are the only materiala they receive.

With that said, I was, and still am, against open book testing for cadets.

[manfredvonrichthofen]

I think that my biggest pet peeve about online testing is that I have heard Senior Members speak of online testing as such a wonderful thing because now they don't have to take the time to grade the tests.

Hey, you said you would do the job, now if you don't want to really do the job, then don't say you will do it.

[SarDragon]

I'm mixed.

WIWOAD, I took correspondence courses that had open book testing, but we were responsible for knowing the info when we took our promotion exams. The analogy here would be the achievement tests, and the milestone tests. We did OK with that system. There needs to be a minimum amount of learning in there.

OTOH, I prefer the human interaction of grading a closed book test. I think the feedback is a better quality than what shows up online.

YMMV.

[jeders]

First of all, Milestone awards like Mitchell and Spaatz are still paper based closed book. They are actaully the relief valve on the whole online process. If the open book tests really do make it easier for cadets not to learn the material, then we'll see fewer people with these awards and they'll become more coveted I think. But I also don't think we're going to see a huge drop off either.

Next, I was a Mitchell cadet BITD (it was a Tuesday by the way) and am now a DCC. I've taken the new open book Leadership tests with the book open in another screen. I failed Ch1 twice (a 76 and a 72) before finally passing. I can tell you that it's not as easy at it seems on the surface.

My biggest pet peeve with online tests isn't the tests themselves, it's that with the ability to take tests at anytime, my "older" cadets still don't take 30 minutes to do it. So we still offer paper testing at the unit which negates the whole "don't have to grade/store tests" arguement for us.

Do I think that online testing is the best solution, no. Do I advocate it, yes, because that's the direction that the world and CAP both are headed.

[davidsinn]

I love the online testing because the questions are a lot more comprehension based and less regurgitation based. I like how the cadets can test more often instead of once a month. I have a cadet that is rather bright that had to take the new Curry three times before he got it so it's not that easy. I like how the test is different every time so it eliminates taking the test and then looking up the answers and only learning the answers to the test. That means the cadets have to have a better grasp of the material.

[manfredvonrichthofen]

I know it can be easier than many think. All you need is Acrobat, and the manuals in PDF. Open the test, and open Adobe, type in the key words into the search bar in Adobe and you have your answer within two to three presses of the enter key. It really teaches nothing. I would think that were cadets to take open book tests then you could go over some of the material in the manual during a review board. However that is not the case.

I am glad that there is still a requirement of in person closed book testing for milestone testing.

[coudano]

If we leave it up to them to take the test online with an open book, then they will be ill equipped to lead when their time comes.

Do you really think that the leadership textbook equips cadets to lead, when their time comes?
Do you see cadets retaining what is in the book, and trying to apply it in practice?
Because I don't.

And that has nothing to do with the new test regime, or the new textbook.  It was true on the predecessor, and even the text from when I was a cadet.

I see cadets taking the leadership tests as a hoop to jump through for promotion, and I see them memorizing and regurgitating for an achievement test, and then flushing; then re-memorizing, re-regurgitating, and re-flushing on the milestone.

That's just my experience though, others may see different trends.


If the prestige of the milestones is reduced by online testing (which is debatable, i'll argue that fewer cadets will get to the milestones now than before) that reduction in the testing component of the prestige is and will be replaced by an increase in the importance of cadets demonstrating /actual leadership skills/ according to the phase progress chart in 52-16, as evaluated by the commander.

[manfredvonrichthofen]

If we leave it up to them to take the test online with an open book, then they will be ill equipped to lead when their time comes.

Do you really think that the leadership textbook equips cadets to lead, when their time comes?
Do you see cadets retaining what is in the book, and trying to apply it in practice?
Because I don't.

And that has nothing to do with the new test regime, or the new textbook.  It was true on the predecessor, and even the text from when I was a cadet.

I see cadets taking the leadership tests as a hoop to jump through for promotion, and I see them memorizing and regurgitating for an achievement test, and then flushing; then re-memorizing, re-regurgitating, and re-flushing on the milestone.

That's just my experience though, others may see different trends.


If the prestige of the milestones is reduced by online testing (which is debatable, i'll argue that fewer cadets will get to the milestones now than before) that reduction in the testing component of the prestige is and will be replaced by an increase in the importance of cadets demonstrating /actual leadership skills/ according to the phase progress chart in 52-16, as evaluated by the commander.

I do see and I have seen different trends. I was a cadet, I was only able to get to the Mitchell before I left for the Army. But, I did learn a lot from the manuals, and I even remember some of it now. I used the information in the leadership manuals when I was a squad leader at the rank of E-3  during the standup of the 506 Infantry when it returned from Korea back to Fort Campbell. I also used it when the unit was at full strength and I was an E-4 team leader. I also used it during the Soldier of the month boards that I won. I was asked what my definition was of leadership, and I told them that my definition was "Leadership is the process of influencing others to achieve a common objective goal or mission." That is also the definition from chapter five of the manuals in 2000. I still lead the same way that I learned to as a cadet and it does pretty [darn] well. Testing isn't about memorizing vommiting back up and brain dumping. It is about learning and having the ability to retain and hand the information off again to another who comes along, along with some of your own personal experience.

[Eclipse]

Actually, testing is exactly about memorization and brain dumping.

Learning and passing on is mentoring, something hard to quantify and harder to test.

There has to be some standard as a gatekeeper, and the military has been using online testing centers for years.

The subjective lessons of followership, leadership, and teamwork will never be found in the books, but we, as a society, are moving from a memorization and repetition based education system, to a reference-based system, meaning you don't necessarily need to memorize the times tables when everyone has ready-access to calculators.  How you impart the fundamentals of multiplication is a separate argument.

What purpose does it serve to know the full military chain of command, up to POTUS (I know, I know), when the information is available, when really needed, with a click?  Does a cadet really need to know the name of the Region CC, someone he is likely never to encounter and if so will introduce himself?  Does it enhance his experience to be able to recite all the members of the Apollo program?

I know that Linebacker I & II were anecdotally interesting when I took ECI13, but they mean zero in terms of leadership lessons or my CAP experience as a Field Grade Officer / Commander, but I needed to "know" things about it to get through the test.  How about the heel height or skirt length for a female member?  Its required knowledge for organic CAP Captains, but useless unless you are a female member or being asked the question, at which point I can ((*click*)) and give the most current answer.

Wailing about the way the world worked WIWAC won't change the way things work today.  You made it to Mitchell and used the lessons you learned at the level for which they were intended, I hope our cadets do the same and more, but I doubt the thing which made the most impression was what you learned in the manuals.

I don't like it, I think we should find a way to use closed-book tests with proctors, but it is what it is, and from my unit CC's who are using the new system,
I haven't heard any complaints about "dumber" cadets - the only challenges they have are related to the "tweeners" or those with less ubiquitous access to the net, which can be easily re-mediated.

[Briski]

I see cadets taking the leadership tests as a hoop to jump through for promotion, and I see them memorizing and regurgitating for an achievement test, and then flushing; then re-memorizing, re-regurgitating, and re-flushing on the milestone.

Cadets have always done this, and they always will. It's a skill they master in school.

I still lead the same way that I learned to as a cadet and it does pretty [darn] well.

I don't. I think I know what you're getting at, but the point is... well, I look back on the stuff I posted here and at CadetStuff.org as a Cadet NCO in 2003 and as a new Mitchell Cadet in 2004, and it's pretty interesting to see how my perspective has changed over time.

I don't know how I would have felt about online testing back then, but I know that I've developed more emotional intelligence, I've become a better communicator, and my critical thinking skills have improved.

Testing isn't about memorizing vommiting back up and brain dumping. It is about learning and having the ability to retain and hand the information off again to another who comes along, along with some of your own personal experience.

I hate to admit this, as a professional educator... but that is exactly what tests are for. The educational paradigm in the United States totally misses the point. This is not a jab at the public school system, but rather a critique of the education system as a whole. The natural consequence of not memorizing and regurgitating is a failing grade, whether in CAP or school.

I didn't learn how to lead from the textbook I read as a cadet. What I gained was a new language to describe the behavior I saw on a weekly basis at squadron meetings, and especially at longer activities such as encampment.

Testing isn't about memorizing vommiting back up and brain dumping. It is about learning and having the ability to retain and hand the information off again to another who comes along, along with some of your own personal experience.

(Quoting this part again because of the second idea...)

Leadership textbooks are inherently abstract. They discuss theories and philosophies in a way that isn't readily applicable to real life. Brain research indicates that adolescents are still developing the ability to think abstractly, which is where the challenge comes in. My personal understanding of my role as a CP Officer is to help bridge that gap—particularly when their behavior doesn't line up with what they say they believe in. Yes, a lofty goal like this is easier said than done.

This is the real test. Cadets need to learn the language of leadership through academics: the leadership text, weekend leadership academies, RCLS, COS, etc. But cadets also need to learn what those words look like in real life: holding positions at the local squadron, serving on staff at encampment, color guard and drill team, etc.

If either of these two elements is missing, the leadership training is incomplete. The testing program ensures that the cadets learn the language, but the true test comes when they're actually put in charge of something (and someone).

There's a little nugget in the margin in Leadership 2000 and Beyond, Vol. 1. I think it's chapter four or five. Anyway, it says that it's more important to know how to find the information than it is to know the information. With technology, this statement is proving to be more applicable all the time.

If cadets try to breeze through the program without learning the language of the leadership curriculum, it will catch up to them when they come to Milestone Awards.

And that presents its own lessons about life and leadership.

[CAPSGT]

 :clap: :clap: :clap: To Briski

I don't have the actual numbers handy, but my DCC did an analysis and found the passing rate to be significantly lower with the online testing.  A lot of cadets seem to be timing out when taking the online tests, so I don't think it's quite so easy to simply Ctrl+F to find the answer as some might think. 

My experience over the years has been that a majority of the cadets who passed the old tests with perfect scores and promoted in minimum time were great at memorizing and reciting passages from the book, but when it came to applying the concepts couldn't even lead themselves down the hall.  If the new tests are more comprehension based (which hinders a search function), that means the new tests are certainly going down the correct path.

The point about the time factor is also in part a valid one as well.  In a unit with a bare bones senior staff where everyone wears at least 5 hats, reducing the workload of grading tests can help to focus the attention of testing officers to other responsibilities they may have as well.  If the cited comment was made along the lines of "good now we don't have to grade them so I can just kick back" that's a different story.  The more important time aspect with online testing though is that it opens up more time during squadron meetings for valuable activities that all cadets can take advantage of, such as practical leadership exercises or Aerospace activities to reinforce what the cadets are being tested on.

One final point that I haven't heard mentioned yet is that by forcing cadets to take the tests on their own time, they are getting a practical lesson in taking the initiative as well as time management.  While they really shouldn't, we all know that there are cadets who would walk into the squadron meeting, not even realizing that testing is going on and take a test regardless of if they had studied for it or not.  By forcing them to schedule time for testing themselves, they must now put some effort forth.

YMMV

[Ron1319]

I was going to write a response that said everything Maj Crockett (CAP) said, so +1.

[lordmonar]

My view on the online testing is that it frees up a lot time to do real training.

The cadets can test when they are ready and as often as they like.

If you are like my squadron....we test once a month.  A cadet fails a test he has to wait 30 days before resting.....now....they can restest in only 7.

As far as using the tests a measure of readienss for promotion.....I think it ranks pretty low IMHO.  If a cadet who has passed a certain test fails to apply that knowledge to his/her duty performance in the practical leadership lab of everyday CAP life.....then he/she will not be promoted.

So....yes cadets can do a word search and pass the test.......how is that any different then any of the tests that seniors have been taking for the last 5+ years?

ES/FEMA/SAFETY/Cadet Program/et al....have all been on line for a long time and are all open book.  Why a higher standard for cadet testing and not for the others?

In my 22 years of AD military......I have had to memorize a lot of trivia to get promoted.  But I did not learn those leadership concept from the PFE texts.  I learned them from doing it.  I learned them from watching my supervisors doing it.  When I went to USAF PME schools....they did not really teach me anything I did not know before.  At the most....the taught me the terms and theory behind what I was actually already doing as an NCO.

So.....over all I don't really hold the test up as that important in the big scheme of things.  It is a hoop that the cadets must jump in order to promote.....but not the most important one.

[MSG Mac]

I have no problem with on line testing for Cadets (or real people either). The problem is that cadets are assuming that because it's an open book test that they don't have to study for it.It must be emphasized that they have a month to read the applicable leadership subject matter, and the next month to read the AE materials.  Some of my cadets have an ongoing problem with either timing out or not getting credit for the test once they've supposedly passed. In those cases I will print out the test for them and have it done during flight time.

[a2capt]

Of the failures for my unit I'll say that most of those would have passed the prior tests as they were in the mid-60's to below 80.

I've had a couple of the newer cadets comment that the newer texts are actually easier to read/work with to them after having compared with the prior ones since they were some that joined right at or before the transition and had a chance to see both at the beginning.

When the crop of cadets hits the Wright Brothers test I'm sure we'll all have a better idea of how this is going to work.

FWIW, after several attempts at testing with a particular cadet and finally getting to the bottom of it being the IE8 problem where it does not even start, because I showed them how to do it with a projector and 'net connection- "I never saw that!", was what I heard.  So one cadet asked if he could take the test right now, since he'd been ready for it and wanted to do it. I said "yes, but I'll give you the paper version because I need the computer for some other stuff this evening" .. and shortly after I realized that the new printed versions of the tests that I had made, had not migrated into the test box yet but rather were still in their obscure disguised transit method.. sitting in the house with some other papers to be brought in.

I explained the situation and gave them the previous LL1 and said "you can try this one, if you pass it, we accomplish the same thing, if not - I'll arrange to come in earlier next week if you don't get it done on the computer between now and then", he passed the thing at 88% so something worked. 

Of course this isn't indicative of everyone.

Most of the issues I've had were not in passing/failing, but just getting it to work. If they used Mac OS X, there wasn't any issues but Windows.. OMG. Talk about the irony, and with IE8.  Even where there are internet access restrictions in the household, the parents have been supportive and made amends, time, etc- as needed.