SET...what is the best approach?

[Major Carrales]

Many point out that our Skills Evaluator Process is in need of an overhaul.  Currently all one need do is qualify, take a test and (according to NATIONAL) one could be able to "sign off" qualifications.

So...what is the best way to do this?  Now, let's establish some rules for this.

1) Let's make this a "policy writing" sort of thread.  A viable system that can be implemented into a REAL WORLD CAP.

2) This is not a thread to bash anyone or anything.  Once we present some solutitons to improving the SET rubric...let us resolve to solve problems in people's suggestions to make them work.

3) Be not affraid to compromise and combine ideas.

4) While we are trying to make CAP more accountable in its training, threads calling for a total dismantleming of CAP training or supplementing it with things that will take years or decades to implement will serve no constructive purpose.  We can have a transition to these things, but CAP training must go forward.

5) You must sumbit a solution before knocking anyone else's.

Here are some possible solutions...

1)     Unit commanders and OPS officers form a committee to recommend names to the Group Commander on a Quarterly basis.  This unit level committee would follow a “rubric” developed group wide; including a total of five sorties and sign-off from an existing SET.

2)     The Group OPS officer collects information from various candidates based on reports from Incident Commanders, it being the responsibility of the individual “candidate” to submit the info from the IC.  This would require a new WING FORM.

3)     Unit commander recommendation to Group…this may create friction.

[sarmed1]

This is one of the sore points with me for CAP ES.  CAP seemed to flip from one extreme to the other  when we switched from TTT to SET.  Basically a very intensive requirement to almost nothing.

Personally I would like to see something of a swing back toward the original.

In this area there are two distinct facets that CAP needs to look at.

Teaching and Testing.

SET is testing only and I think there needs to be a little more to it than that.  But even with "evaluator only" status there is no training out there on how to test.

The old TTT course had a section on how to evaluate, basically common pitfalls and helpful tips.

Some concerns
I think instructor/evaluator qualification and training needs to stay at a local level, at best no higher than group.  Otherwise you run into the pitfall of before where each squadron had maybe 1 or 2 people that had to be able to sign off everyone, and then you were stuck if you wanted something differant in the way of qualification those guys didnt have.  TT courses were few and far between, so most people gave up or timed out on their training.

Some kind of experience qualifier....1 yrs qualification in any specialty other than GES

Some sort of instructor training/development course...be it in person or online, even if you just want to evaluate.

Pass the online test

Some sort of "...observed once before being freed from the nest..."

mk

[lordmonar]

The easy and best fix in my opintion....is to create an ES specialty called Skills Evaluator.

This would require you to take the SET test (or something like it) and do some reading on instruction/evaluation, and to conduct a class (such as teach a task that you are already qualified on) and evaluate a simulated student (evaluate student on task taught in the class you held)

This would require some sort of practical teach back before the new evaluator/instructor was turned loose on the the world.

I would not require a two sortie requirement but maybe teach and evaluate two different tasks at two different days (you can't teach two related task like "take a bearing" and "follow bearing and locate object").

I would also require all evaluators to be approved in writing and identified in MIMS/E-service as trainers and what areas they are allowed to train in.

This would put the onus of control on the commander.  If Jose ES is a great GTL but has no or is poor at teaching...the commander does not hack off.  If Joe ES is a great instructor, lots of experience as a GTL but is only a so so Observer, then the commander can appoint him a GT instructor but not an Observer/Scanner instructor.

[Eclipse]

The way it is defined in MIMS today is the way it should be done.

Anyone can take the test, but since the Unit CC is the ultimate arbiter of the sign-offs, there should be no chance of P-51 sign offs.

Sure a cadet could knock down all his friends, but the Unit CC can simply disapprove those, with a warning
to not do that again...

In effect the Unit CC would / will maintain a list of unit approved SETs, and anyone else would know not to even try.

Nepotism, favoritism, or incompetence would be handled through existing channels for complaints, etc.

The unit CC's are ultimately responsible for their people, so they should be responsible for their sign-offs, too.

That there is an assumption that abuse will occur is beside the point (and sad).

That higher-HQ staffers believe they can somehow have better oversight of an SET 350 miles away, then their local CC, speaks volumes of the trust in some states.

edit: If there isn't already, there certainly should be a connection between both qualification and currency to allow one to act as an SET (i.e. qualified with x number of mission activity in x time, or the buttons don't work).

edit 2: I like the idea of requiring a year as the specialty before you can be an SET - that eliminates the chance of members signing off their friends the day they get GT3 themselves.

[Al Sayre]

I've got 60 odd people in my squadron, about 1/2 have taken SET and it really hasn't been a problem.  I simply stated my policy that Cadets do not have sign off authority unless specifically authorized by me, and Officers also need to get my approval before they start signing things off for people.  I also review all of the sign offs for my people, and if I see a name or CAPID that looks fishy ( completed at some distant SAREX etc. ) I don't approve it until I talk to that Evaluator or their CC.

[Stonewall]

I may be off here since I haven't done any ES stuff in over 2 years, but if I remember correctly, isn't SET just an online quiz or course or something?

I remember doing mine at work in under 20 or 30 minutes, or was that something else?

If that's the case, then it's a worthless program.  It reminds me of all the online "training" the Air Force has and requires annually.  NO BODY LEARNS ANYTHING!  It is simply a hurry up and get through this so we can put your certificate on file so we're not flagged kind of deal.  No one cares and no one gains anything of value from it.  Like the Air Force Information Assurance course I had to take.  I couldn't tell you a single thing about what I was supposed to learn other than if something happens that involves my LAN account, I need to tell someone.  Did I need to spend an hour [plus] on the computer to learn that?  Nope, would have already done it because it makes sense.

I am NOT a fan of computer based training, especially in the military (that includes in CAP).  Even at SERE school we had several hours of CBT in lieu of actual hands on training/instruction from instructors.  Blew my mind.

[Dragoon]

Since the purpose of SET is to ensure you can conduct meaningful, hands-on evaluation...


1.  The training MUST include a hands-on evaluation of your ability to evaluate.  You should have to demonstrate to an instructor that you can set up and run an evaluation, a correctly passing the folks who need to be passed and identifying (and counseling) the folks who need more training.

Having an online book test for a hands on skill is ludicrous!


2.  Wings (or groups, for big wings) should identify those people who are qualified evaluatiors FOR EACH SPECIALTY seperately.  Just because I'm qualified to evaluation Ground Team skills doesn't mean the minute I get my scanner card I can evaluate other scanners.  We need seperate lists of evalutors for each specialty.

3.  Some specialties should require SET qual.  I can't imaging a GTL who isn't authorized to evalute at least GTM3s.  Ditto a CUL who can't evaluate radio operators.  This pretty much defies logic.

[ZigZag911]

We've been discussing this at Wing HQ quite extensively recently.

Here's the feeling:

1) The electronic version of SQTR needs 'approval' function for entry into advanced training, and for certification of final qualification....these would be squadron commander or designee, with the final cert. flowing up the chain

2) SET in and of itself qualifies an individual to give another training and/or sign off tasks in his/her area(s) of specialization.

3) Final evaluators need to be ES personnel with deeper training and experience in the area concerned; we're discussing what the objective criteria meed to be at this point. We are concerned here with knowledge, training, and experience of the evaluator -- their level of unit assignment is irrelevant (squadron, group, wing).

[ZigZag911]

I am NOT a fan of computer based training, especially in the military (that includes in CAP).  Even at SERE school we had several hours of CBT in lieu of actual hands on training/instruction from instructors.  Blew my mind.

Computer based training can supplement or substitute for classroom time and outside reading.

It must, as you rightly point out, be complemented by practical experience and actual demonstration of skills, live and in person, to a qualified evaluator.

Tags - MIKE

[Psicorp]

What we do (and what I've seen done in other units) is not pay a whole lot of attention as to whether someone is SET qualified or not if the training is done within the unit.  All SQTRs are reviewed by the unit ES Officer.  Data is usually entered into the WMU by the unit ES Officer using his/her CAPID as the authorizing/verifying Officer.   The ES Officer is the person who, once the SQTR is complete, submits the CAPF 100 up the chain electronically as the requestor.

If training is done outside the unit (SarEx, etc.), then the neat thing about those blue and green books is that they also contain questions the trainee should be able to answer if they actually learned the material.  If the ES Officer isn't satisfied that the individual is capable, he/she recommends further training to the CC, who makes the final determination.

This all assumes that the ES Officer is very involved with the ongoing training and training schedules. 

We're soon going away from the WMU and switching completely over to eServices, so things may change.

[floridacyclist]

Once upon a time, FL had a program called the WAT or Wing Authorized Trainer. There was an online list of approved WATs and you had to apply to the wing ES person to be on the list. My older son and I attended a 2-weekend course in Orlando for MRO WAT and it was simply awesome. The first weekend you were the student and evaluatee, and the second you were the teacher and evaluator plus there were classes on evaluation technique (they put those questions at the end of each task for a reason) . It was pretty good stuff that I wish we could see more often in other specialties as well.

[arajca]

What we do (and what I've seen done in other units) is not pay a whole lot of attention as to whether someone is SET qualified or not if the training is done within the unit.  All SQTRs are reviewed by the unit ES Officer.  Data is usually entered into the WMU by the unit ES Officer using his/her CAPID as the authorizing/verifying Officer.   The ES Officer is the person who, once the SQTR is complete, submits the CAPF 100 up the chain electronically as the requestor.

The problem with inputting ES quals in WMU is they will never make it from WMU to MIMS. Which means, as far as National is concerned, you aren't qualified.

[Psicorp]

What we do (and what I've seen done in other units) is not pay a whole lot of attention as to whether someone is SET qualified or not if the training is done within the unit.  All SQTRs are reviewed by the unit ES Officer.  Data is usually entered into the WMU by the unit ES Officer using his/her CAPID as the authorizing/verifying Officer.   The ES Officer is the person who, once the SQTR is complete, submits the CAPF 100 up the chain electronically as the requestor.

The problem with inputting ES quals in WMU is they will never make it from WMU to MIMS. Which means, as far as National is concerned, you aren't qualified.

Right.  Which is one of the reasons we're moving away from the WMU completely, but I don't see our procedure changing, inputs will still be done by the ES Officer.

[ZigZag911]

Once upon a time, FL had a program called the WAT or Wing Authorized Trainer. There was an online list of approved WATs and you had to apply to the wing ES person to be on the list. My older son and I attended a 2-weekend course in Orlando for MRO WAT and it was simply awesome. The first weekend you were the student and evaluatee, and the second you were the teacher and evaluator plus there were classes on evaluation technique (they put those questions at the end of each task for a reason) . It was pretty good stuff that I wish we could see more often in other specialties as well.

Can you get hold of -- or point me in the direction of -- the course syllabus?

[floridacyclist]

Let me check my work computer....I think I have a copy of the CD they gave us on there....will know in a couple of hours.

[floridacyclist]

Sorry, the closest they gave us was a quick 10-slide PPT on MRO...and all it did was show various parts of the CAPF101T (itself obsolete). The rest of the CD was a few more PDFs on MRO, GTM/L and UDFT. All of this info (excep the useless MRO PPT) is available on ndcap.org.

Shame it wasn't better-documented as it was a really good class.

[BillB]

The Florida Wing ES Training Officer may have the syllabus since that was the office that approved WAT personnel. I'll let Gene contact them since I can't  LOL

[floridacyclist]

I'll check the squadron computer Thur night as I think I'm missing some files from the copy of the CD on my work computer....I could have sworn there were some PPTs regarding ACUT, BCUT, Evaluating,  and MRO. If nothing else, I'll see some folks this weekend in Jax.