Spot checks on SQTR task capabilities

[RiverAux]

One of the common knocks about the CAP SQTR system is that despite having very clearly spelled out actions that must be accomplished to complete the task or specific bits of knowledge that must be known, the system still isn't consistently applied everywhere. 

As a way to sort of guard against that and as a general quality control measure, what do you think about having some sort of system to do random spot checks on individuals abilities to successfully perform a few of the tasks in specialties they are qualified in?  To start with, I'm thinking that it might be something to include in the Air Force mission evaluations held every few years. 

It could be as simple as the evaluator going up to a member on a ground team, asking them to step aside for a few minutes and giving them a short verbal quiz on some of the knowledge tasks or could be as elaborate as having a short navigation course set up to run a few ground team members through.

What we would be looking at would be trying to get a good sample of members rather than something that would be done to everybody.

At the end of the evaluation you might end up with a table listing the tasks tested, the number of individuals tested on that task, and the pass/fail rate. 

Of course this could be implemented outside of mission evaluations as well.  Done consistently it could expose those units that are doing a bad job of training people and let you know where work needs to be done. 

I am not proposing that these spot checks make you do everything on the SQTR for that position.  That would just take way too much time to do.  It would be better to test a bunch of folks on a few tasks rather than a few individuals on everything. 

Obviously, which tasks would be tested as being "important" would be a judgment call.  But the good thing is that you can easily switch things up and test other tasks if the word gets out about task x being tested that day. 

[cap235629]

As long as the same principle is applied to aircrews, I am all for it.  I know of a lot more pencil whipped scanners and observers who by the nature of being a pilot are deemed to be qualified and just signed off.....

[Eclipse]

Recurrency on all tasks is required during the period of qualification now, as is mission participation.

The issue with your idea is that it would require "super-SET's" who can simply decide at a whim to re-task someone, just the thing for unnecessary abuse.

I don't want someone outside my AOR re-tasking my people on their whim.  If there's an issue with their capabilities it should be handled through the chain. 

If consistency in application is the issue, address it there.

[RiverAux]

The issue with your idea is that it would require "super-SET's" who can simply decide at a whim to re-task someone, just the thing for unnecessary abuse.

Gee, I don't recall suggesting anything of this nature.  In fact, I suggested we start with having it done by AF types during the mission evaluations.....

I don't want someone outside my AOR re-tasking my people on their whim.  If there's an issue with their capabilities it should be handled through the chain.

See thats the thing --- we've got an incestuous system where it is entirely possible for a CAP member to only operate with people from their unit who signed off on their initial training and requalification through their entire CAP career.  That is a recipe for developing pockets of folks who are not really up to the standard since no one else ever looks at their individual capabilities. 

Really these spot checks are designed to spot unit training failures more than as a witch hunt to find unqualified members.  If you test 10 GT members from one unit and only 2 of them managed to pass a spot test of some bit of required knowledge while in another unit 7 members passed, I think you've identified a unit that has some sort of problem.  Then it is the leadership's job to figure out where they've gone off track and to fix it.

As long as the same principle is applied to aircrews, I am all for it

Sure.  Same principle -- easy to spot check the knowledge tasks, logistically harder to spot check the field tasks. 

[Eclipse]

(Why don't your quotes have the proper references so they can be clicked back?)

1. Most "AF Types" don't have the detailed program knowledge themselves to be able to spot-check CAP members, especially on
something outside their personal expertise.

2.  The system isn't "incestuous", that's how its designed.  Commanders are responsible for the training of their people, the approval of the training, and remediation of the problems.  The reason we have to go trotting all over the wing to get trained is a by-product of being short-handed, not the intention of the system.

3. The responsibility of insuring plans are up to snuff is with the next higher HQ, not some random uber-SET who decides to give someone a hard time.

[SarDragon]

(Why don't your quotes have the proper references so they can be clicked back?)

Because he isn't copying the whole quote tag for the subsequent sections.

Tag 1 looks like this:

Code Select Expand

[quote author=Eclipse link=topic=9735.msg176090#msg176090 date=1264047477]

Tags 2 and 3 look like this:

Code Select Expand

[quote]

[Short Field]

RiverAux - I am with you 100% on this one.   :clap:  :clap:  :clap:

[RRLE]

It could be as simple as the evaluator going up to a member on a ground team, asking them to step aside for a few minutes and giving them a short verbal quiz on some of the knowledge tasks

One problem with that is that many knowledge tasks built into qualifications (meant generally not just CAP) were put in years ago and long outlived their usefulness - if they were ever useful to start with. The candidate mentally bit-buckets the less then useful knowledge as soon as the exam is over. What is retained is what is actually needed to do the mission.

Your proposal would require trained and productive members to retain useless info forever just to satisfy some higher-up on a tear or worse a personal vendetta against the member, his unit or the member's immediate leadership.

[wuzafuzz]

Instead of opening the doors to perceived harassment of folks in the field, why not just require sufficiently challenging exercises at SAREX's?  Call it good if your teams can pull it off.

[RiverAux]

One problem with that is that many knowledge tasks built into qualifications (meant generally not just CAP) were put in years ago and long outlived their usefulness - if they were ever useful to start with. The candidate mentally bit-buckets the less then useful knowledge as soon as the exam is over. What is retained is what is actually needed to do the mission.

Your proposal would require trained and productive members to retain useless info forever just to satisfy some higher-up on a tear or worse a personal vendetta against the member, his unit or the member's immediate leadership.

I think we're going to have to disagree on this one because with a few exceptions I think the knowledge tasks that are part of CAP's SQTR system are relevant to mission capability. 

3. The responsibility of insuring plans are up to snuff is with the next higher HQ, not some random uber-SET who decides to give someone a hard time.

Please describe to me CAP's current system by which Wings are evaluating the effectivness of squadron-level ES training as it relates to individual skills.....there is none.  The current Mission Evaluations are almost entirely focused on mission planning and generally barely touch on individual skills. 

Where is this super-SET stuff coming from.  I didn't propose that. 

[Short Field]

Where is this super-SET stuff coming from.  I didn't propose that.

LOL.  I always thought the SUPER-SET was the member with SET quals who just barely got signed off and then proceeds to sign off everyone willing to listen to him expound on his great expertise for 10 minutes.   Thankfully, they changed that in the last 60-3!!! 

[DG]

RiverAux,

I can tell you that National is working on this specific issue.

It may start on the air side.

All Mission Check Pilots will be required to undergo an intensive reevaluation, like an audit.  Reevaluation by a National Evaluator from outside their Region.

Those who know their stuff have nothing to fear.

I can see how those who are incompetent and don't want to be found out, will resist and fear this program.

[Eclipse]

Where is this super-SET stuff coming from.  I didn't propose that.

Who else would do it?

Not the USAF, they don't have the program knowledge.

Not some random SET outside the chain of a respective member.

Who, then, other than Wing-approved SME's?

[Eclipse]

All Mission Check Pilots will be required to undergo an intensive reevaluation, like an audit.  Reevaluation by a National Evaluator from outside their Region.

That will never happen.

It has nothing to to with fear of being "found out" and everything to do with just being able to keep people current, no consistency between wings, let along regions.

If you have issues with training proficiency, address it through better enforcement of existing programs and creation of some sort of standard which must be adhered to.

As it stands today, we have "NESA people" and "Non-NESA People", and when they work together they fight about minutia, and technically they are both right. And that's just two of the multitude of "camps" we have.

Start there and forget about "national evaluators" unless you really do want to have no crews (ground or air) left.

[RiverAux]

Where is this super-SET stuff coming from.  I didn't propose that.

Who else would do it?

Not the USAF, they don't have the program knowledge.

Not some random SET outside the chain of a respective member.

Who, then, other than Wing-approved SME's?

Someone outside that persons chain of command is EXACTLY who needs to be doing it.  It is that independent evaluation that we'd be looking for. 

I'd say that the Wing and/or Group ES officer and their assistants would be the proper ones to do this.  Since the idea is to conduct spot checks rather than trying to evaluate everyone, not many people would be needed.  Really anyone qualified in that position and with SET (a joke, I know, but its what we've got) could do it. 

[Eclipse]

^ those people are already charged with supervising the SET's in the Group and Wing, and if they aren't active enough to
be seeing their people perform, especially at the Group level, they shouldn't be in the job to start with.

[RiverAux]

^ those people are already charged with supervising the SET's in the Group and Wing, and if they aren't active enough to
be seeing their people perform, especially at the Group level, they shouldn't be in the job to start with.

Again, please describe to me CAP's system for performing this supervision.  What I am proposing is a formal way of doing it.   

[Eclipse]

Seriously?

Every task has to be approved by a commander, and upgrades by a Group CC.

The commanders and ES staff are charged with insuring people are being pencil whipped and are capable.

If you think they aren't in your AOR, address it there.  We don't need the SET-Stasi or evaluators from other regions making it
harder to stay current, or adding another hoop to an already sometime onerous process.

[heliodoc]

Is there a "secret" CAP Stan Eval team out there going to do this?

Months ago I read here CAP MAY lose approx 4-5 million in funding...

I will believe when I read and see it come down officially.  But it would be a welcome "operation" if the credentialing issue is finally hitting the dirt for DHS and HLS type missions.  Who knows? 

Will it be a super duper team from the NHQ, Region, or Wing?  Are they going to be SAR gods from NESA only doing this?

CAP, with all its zeal, going from NO standardization to "all of a sudden" going into crisis mode and trying to reevaluate MCP's?

That wilL be a project ...how much is that going to cost?  Is this "ReEVAL program going to exceed current CAP or FAA or other standards?

I got nothing to fear.. I know my proficiency level is pretty low right being on layoff...but next month I am going to get me a real BFR and IPC outside of CAP.  So I am ready for real flying up and coming.

This so called ReEval team should be fun to watch come together ....especially when its mentioned on CAPTalk...where you hear it here first without any substantiated evidence.  More fun to watch going from  NO standardization  US CAP wide to "this will be accomplished in 1 year" type of thing.  Is this outfit and those "selected" individual(s) up to the task?  Where are they getting alll that vaca time.  How about getting few of us  laid off pilots to do this?   We are the true non biased individuals..  how can any of those folks going from Region to Region , Wing to Wing be Non biased?  Train us laid off  pilots at a RELIABLE CONTRACTOR outside of CAP.  Get a CONTRACTOR with a TRUE SAR syllabus (AF) or AF contracted to do this.  Then we will see some TRUE and non bias in the Training and ReEVal program...think that will be cheeeep?  CAP won't be any cheeeeper!!  Cus the standardization does not really exist now and it WILL take more than 2-3 years to accomplish.  If it is gets "accomplished' in one year...I would be looking at the reliability of the training especially when some of the curriculum is so weak now.  CAP NHQ got a REAL training program together on the level of Part 121 and Part 135?   When CAP training appproaches that...I may believe in it!!

But some CAPers MAY claim they got the Pulse on NHQ and everything CAP

[Al Sayre]

If your IC's are doing mission critiques properly, you get re-evaluated every time you fly a mission.  As an IC who runs about 1/2 the wing SAREX's, I know which MP's & crews can consistently find the target in a minimal amount of time, and which ones don't know their equipment and wander about and either don't find it, or just stumble across it.  I usually try to give them some coaching and point them back to the training materials.  There's more to being a MP than being able to fly the search patterns and pass the CAPF 91...

[RiverAux]

Al probably represents the best case scenario, but I'd say that he is probably pretty unique if he runs that high a percentage of the missions in his wing.  In my opinion the system has failed if we're depending on mission staff to identify such weaknesses since there is a pretty big gap between where the staff sits and the individual scanner or ground team member is out in the field. 

The commanders and ES staff are charged with insuring people are being pencil whipped and are capable

Once again, HOW are they supposed to be doing this?  Keep in mind that most Wings don't have groups and the Wing ES Officer is very unlikely to know the individual capabilities of all ES qualified people in the Wing. 

Right now, we are trusting the SETs to do their job, but we are not VERIFYING that they are. 

[Short Field]

As an IC, I have identified MPs to the Wg Ops staff that failed to follow the mission plan on SARs.  One simply decided that his crew couldn't see anything on the due to "rough terrain and trees" so he decided to do a ELT search instead of the grid search he was task.  Another came back as his crew were complaining that he never attempted to fly a search grid or drop to search alititude.  He also couldn't find the practice target at a following SAREX (which stood out if you got within five miles of it).  Wing's response:  They are our best pilots so we can't do anything to them...

[JoeTomasone]

FLWG began requiring that all echelons maintain a copy of any SQTRs a year and change ago so that in the event a qualification came into question, the signed SQTR form could be reviewed, and the SET who signed off on it could justify it.    When I was a Group ES Officer, I took scanned SQTRs via email into a special email list that went to the two approving entities on Group staff (myself and my Training Officer) and to a Gmail account for permanent/backup archival.

What I like about this system is that the "overhead" is minimal, and the message is clear - if Lt Snuffy can't read a map, and you signed off on his map reading tasks, you better have a good explanation -- and you better be prepared to requal for that task yourself.    And if YOU can't do it, then we check YOUR SQTR, etc, etc, etc...    I would have had no problem suspending the SET qual for anyone who seemed to be lacking the required judgment. 

[DG]

All Mission Check Pilots will be required to undergo an intensive reevaluation, like an audit.  Reevaluation by a National Evaluator from outside their Region.

That will never happen.

It has nothing to to with fear of being "found out" and everything to do with just being able to keep people current, no consistency between wings, let along regions.

If you have issues with training proficiency, address it through better enforcement of existing programs and creation of some sort of standard which must be adhered to.

As it stands today, we have "NESA people" and "Non-NESA People", and when they work together they fight about minutia, and technically they are both right. And that's just two of the multitude of "camps" we have.

Start there and forget about "national evaluators" unless you really do want to have no crews (ground or air) left.

Talk to your Wing DO.

Ask him if it isn't so.

And, of course, NESA materials and standards will be codified as the National Standard.

[RADIOMAN015]

What I like about this system is that the "overhead" is minimal, and the message is clear - if Lt Snuffy can't read a map, and you signed off on his map reading tasks, you better have a good explanation -- and you better be prepared to requal for that task yourself.    And if YOU can't do it, then we check YOUR SQTR, etc, etc, etc...    I would have had no problem suspending the SET qual for anyone who seemed to be lacking the required judgment.

So how many times did "group" have to get an explanation from the evaluator when one of his/her trainees "forgot" how to do something.  Don't know about anyone else BUT it seems to me that volunteers can "forget" how to do something after being trained by a very competent trainer & evaluated by a strong evaluator, especailly if a particular task hasn't been performed in a  period of time. 

Perhaps a better approach would be prior to any mission having a short review on applicable tasks that the applicable teams might encounter and also carry appropriate checklists and/or reference materials for a quick 'read up" if required.   Even the military has checklists that are REQUIRED to be used during critical operations.

RM

[DG]

Perhaps a better approach would be prior to any mission having a short review on applicable tasks that the applicable teams might encounter and also carry appropriate checklists and/or reference materials for a quick 'read up" if required.   Even the military has checklists that are REQUIRED to be used during critical operations.

RM

Good thinking!

You certainly would pass any EVAL I conducted.

[JoeTomasone]

So how many times did "group" have to get an explanation from the evaluator when one of his/her trainees "forgot" how to do something.
  Don't know about anyone else BUT it seems to me that volunteers can "forget" how to do something after being trained by a very competent trainer & evaluated by a strong evaluator, especailly if a particular task hasn't been performed in a  period of time.

Never while I was on the staff.   It was also subjective, meaning that had I been made aware of an issue, I would have had to make a judgement call as to whether it was pencil-whipping or not.   I could simply ask for effectively a re-qual of each task until I got a sense of what the true story was.  And, of course, Wing could have gotten involved if they felt the need to - which would have acted as a reminder that burying something under the rug wouldn't have been a grand plan on my part.   All in all, not a bad system.

Perhaps a better approach would be prior to any mission having a short review on applicable tasks that the applicable teams might encounter and also carry appropriate checklists and/or reference materials for a quick 'read up" if required.   Even the military has checklists that are REQUIRED to be used during critical operations.

I like that idea as well.   Also great for any trainees that might be accompanying you.

[Check Pilot/Tow Pilot]

Perhaps a better approach would be prior to any mission having a short review on applicable tasks that the applicable teams might encounter and also carry appropriate checklists and/or reference materials for a quick 'read up" if required.   Even the military has checklists that are REQUIRED to be used during critical operations.

RM

Good thinking!

You certainly would pass any EVAL I conducted.

Great idea and something that I will work on in my unit.

[Al Sayre]

I'm not so sure about that,  "Just hold on Mr. IC, as soon as we get done reviewing these 200 pages of task descriptions we'll be ready to jump in the airplane/van and go..." 

You should be fully familiar with your possible tasks long before you show up at the Mission Base.  If you want to spend your meeting time doing that the week before the SAREX great, I'm all for it.

However, because of the minimal training time availabe at SAREX's I don't think that it's a very wise expenditure of time that should be spent showing a sense of urgency in getting the mission accomplished.  The GTL has a checklist to make sure his team is prepared, if they're not prepared, then they shouldn't be there, plain and simple.  Ditto for the aircrews.

The time you waste is NOT your own.  I don't think I even need to go into the time press of a real mission.  Do you want to make the phone call or visit?   "I'm sorry Mrs. Smith, if we could have only got there 2 hours sooner, but we had to review our task guides..." 

[Check Pilot/Tow Pilot]

 I should have said "Great idea and something that I will work on in my unit, at my unit",  Not at a SAREX or RWM.

[Short Field]

However, because of the minimal training time availabe at SAREX's I don't think that it's a very wise expenditure of time that should be spent showing a sense of urgency in getting the mission accomplished. 

Mission on a SAR:  Save Lives.  Mission on a SAREX:  Train & practice how to Save Lives.   If you don't have time to train on a SAREX, you shouldn't have one.  I see too many people show up just for the free flying and to get someone two missions so they can be signed off.  Their objective is just to log two missions, not train someone or get trained. 

[davedove]

However, because of the minimal training time availabe at SAREX's I don't think that it's a very wise expenditure of time that should be spent showing a sense of urgency in getting the mission accomplished. 

Mission on a SAR:  Save Lives.  Mission on a SAREX:  Train & practice how to Save Lives.   If you don't have time to train on a SAREX, you shouldn't have one.  I see too many people show up just for the free flying and to get someone two missions so they can be signed off.  Their objective is just to log two missions, not train someone or get trained.

There are different levels of training.  At a SAREX the focus should be on coordinating units and the like.  Individual training is more a unit level activity.

Yeah, if you have the time, you can do individual training at a SAREX, but that's not really the prime consideration.

[Al Sayre]

You beat me to it.  The SAREX is where you demonstrate your skills and ability to work as part of the greater team.  If your team/squadron has a couple of trainee's along to get a sign-off or two, I'm fine with that.  If an individual wants to train for a Mission Base Position and has completed the pre-requisites, and if the actual person doing the job has time to act as SET, I'm ok with that.  If you show up with 20 unqualified GT or Aircrew personnel and expect to get a full classroom training session and then do field work, I'm not going to do it or allow it.  It would take away from the training and practice of the entire wing.

[davidsinn]

There are different levels of training.  At a SAREX the focus should be on coordinating units and the like.  Individual training is more a unit level activity.

That's all well and good if you have someone to train you at the unit. Or even at a neighboring unit. I live in a nearly ES dead area and can only get training at SAREXs.

[Eclipse]

You beat me to it.  The SAREX is where you demonstrate your skills and ability to work as part of the greater team.  If your team/squadron has a couple of trainee's along to get a sign-off or two, I'm fine with that.  If an individual wants to train for a Mission Base Position and has completed the pre-requisites, and if the actual person doing the job has time to act as SET, I'm ok with that.  If you show up with 20 unqualified GT or Aircrew personnel and expect to get a full classroom training session and then do field work, I'm not going to do it or allow it.  It would take away from the training and practice of the entire wing.

Who-whee +1 and a 1/2!

An eval, GTE, large-scale exercise, and real-worlds is not the place for people to be showing up and asking
"What's a 101 card?"  I have no issue with informed trainees, but basic "What's ES?" training belongs at the home unit, whether
that means actually at the unit, or through activities coordinated with external resources, separate from SAREX's.

[Eclipse]

That's all well and good if you have someone to train you at the unit. Or even at a neighboring unit. I live in a nearly ES dead area and can only get training at SAREXs.

As long as you have a task guide, you can train yourselves.  All you need is coordination with outside resources to get some active members running as SET's, so you can be self-sufficient.

[Check Pilot/Tow Pilot]

Last weekend SQ68 conducted a training SAREX at Hemet, CA where the aim was to provide the opportunity for Mission Pilot trainees (and MO, MS, Base Staff, etc) to achieve as many of their SQTR's as possible and to complete two missions.  It was conducted over a two day period and appeared to be a great success.

Having the flexibility to conduct one of these on a yearly basis assists members in completing SQTR's.  As we know the training time for MP is long and involved, and if this helps achieve our mission then I'm all for it.

Besides, it helped me achieve my Mission Pilot in one year and eight days of membership    I know, not a record, but it sure felt good to me 

[Short Field]

If you can't plan a SAREX to include an extra hour for the crew to cover the advanced task items for the new scanner/observer/mission pilot, you need to reconsider your training objectives.  I see too many "how fast we can get out the door" SAREXs.  Lots of sign-offs, very little real training.   Ground teams don't move near as fast so you shouldn't have the same problems.  Putting a little more spacing between sorties can't help but improve the mission base training.  I am not talking about trying to conduct a full Observer/Pilot Ground School at a SAREX - but training is a major part of conducting a SAREX.   The purpose of a USAF Evaluation is to determine how well the Wing has spend the training dollars it was given over the previous two years.  I sure don't see any training dollars spent EXCEPT on SAREXs. 

[RiverAux]

Drifting.....

[Al Sayre]

If you can't plan a SAREX to include an extra hour for the crew to cover the advanced task items for the new scanner/observer/mission pilot, you need to reconsider your training objectives.  I see too many "how fast we can get out the door" SAREXs.  Lots of sign-offs, very little real training.   Ground teams don't move near as fast so you shouldn't have the same problems.  Putting a little more spacing between sorties can't help but improve the mission base training.  I am not talking about trying to conduct a full Observer/Pilot Ground School at a SAREX - but training is a major part of conducting a SAREX.   The purpose of a USAF Evaluation is to determine how well the Wing has spend the training dollars it was given over the previous two years.  I sure don't see any training dollars spent EXCEPT on SAREXs.

We have put on both MS & MO schools. The MS school was well attended ~20 people, the MO school only had 6.  If folks can't be bothered to take advantage of the school opportunities when presented, they shouldn't expect the Ops/ES staff to carve out a significant portion of the training time and budget to accomodate them.  We have a whole wing to consider.

[arajca]

If you're going to spot check sqtr task capabilityies, make sure you use the task evaluation form in the task books.

[RiverAux]

Definetely, and it would be closed book too (which is how you're supposed to be evaluated though some hear have said that some places are letting you use the task guides while being evaluated). 

[Eclipse]

Definetely, and it would be closed book too (which is how you're supposed to be evaluated though some hear have said that some places are letting you use the task guides while being evaluated).

Cite please.

Nothing wrong with using the task guide, its required equipment.

[RiverAux]

Already discussed to death in another thread and too likely to lead us far off topic for further discussion in this one.  Necropost the other thread...

[Eclipse]

Well why insert an opinion like that in your own thread if you don't want it to be challenged?

[arajca]

Definetely, and it would be closed book too (which is how you're supposed to be evaluated though some hear have said that some places are letting you use the task guides while being evaluated).

Each task evaluation section lists what can be used. Most include the task book, therefore using it on those tasks where permitted is fine.