Metrics

[FlyTiger77]

Has anyone developed a comprehensive suite of metrics to monitor the health of (a) squadron(s)?

I trying to determine what the critical information is to monitor all aspects of a CAP unit.

If you would care to share, I would be interested in seeing it.

[a2capt]

One could argue that the items tracked for the Quality Cadet Award are indicators of activity, progress, and the like.

Though they've not been as forthcoming this year so far with the quarterly data.

[Eclipse]

I think this is huge, and CAP essentially ignores this kind of data to their detriment.  The trouble is that the history is nearly impossible to get, so
it can take a couple of years+ to get meaningful numbers.

Unit strength.

Member churn.

Professional development completed / eligible / late.

Yeager completion (should be 100%)

Time it take new members to:
     Join (after first meeting)
    Complete Level I

ES ratings completed / open / expired

Cadet promotions completed / eligible/ late

Meeting attendance.

Other activity attendance.

Squadron of Merit Standing

Quality Cadet Unit Standing.

Individual member rankings in terms of PD/ES/Activities/meeting participation

[A.Member]

PM Sent

[FlyTiger77]

I think this is huge, and CAP essentially ignores this kind of data to their detriment.  The trouble is that the history is nearly impossible to get, so
it can take a couple of years+ to get meaningful numbers.

Unit strength.

Member churn.

Professional development completed / eligible / late.

Yeager completion (should be 100%)

Time it take new members to:
     Join (after first meeting)
    Complete Level I

ES ratings completed / open / expired

Cadet promotions completed / eligible/ late

Meeting attendance.

Other activity attendance.

Squadron of Merit Standing

Quality Cadet Unit Standing.

Individual member rankings in terms of PD/ES/Activities/meeting participation

What formula would you use for "Member churn"?

Automated metrics that can be populated from CAPWatch or some other database without having to hand jam a bunch of numbers would be most excellent.

[Eclipse]

What formula would you use for "Member churn"?

Automated metrics that can be populated from CAPWatch or some other database without having to hand jam a bunch of numbers would be most excellent.

Yes, they would, but good luck with that.

I don't really have a formula, but I'd say starting with anyone who quits within the first 6 months to a year would be considered churn.
And something reasonable for participation probably needs to be included (empty shirts should not be considered in your active numbers, etc.)

That's the problem with the way many commanders (and apparently NHQ), credit themselves with member strength.  They wear the raw number
with no accounting for churn.  A unit with 50 members at the beginning and end of a fiscal year might be seen as stable, until you look
deeper and see they lost 5 experienced members, and churned 15 within the same year, only to wind up with the same number.

It might be interesting to assign some sort of points value to a member's accomplishments, longevity, participation, etc.  Unless it was done
at a wing-level, it'd be arbitrary to the unit, so difficult to compare (though we're really talking about usable single-unit metrics, so that's where
it's important), but I bet an actuary could knock out something reasonable in a few hours.

Certainly a 10-year member with Level III, 2 ES ratings, active participation, and multiple PD ratings is more valuable to the organization
then someone with 20 years in who still has gold bars and shows up twice a year.

[lordmonar]

Certainly a 10-year member with Level III, 2 ES ratings, active participation, and multiple PD ratings is more valuable to the organization
then someone with 20 years in who still has gold bars and shows up twice a year.

I would say certainly a 3 moth member whit level I ACTIVE PARTICPATION is more valuable then anyone else with all the PD/ES/ST ratings who only shows up twice a year.

[FlyTiger77]

I don't really have a formula, but I'd say starting with anyone who quits within the first 6 months to a year would be considered churn.
And something reasonable for participation probably needs to be included (empty shirts should not be considered in your active numbers, etc.)

In the past I have used mean/median longevity as a rough indicator. As it moves to the right, the unit is more stable. It is subject to the influence of a mass influx of members, though, and also gives the 20-year member who only shows up when we are turning avgas into noise the same credit as the member that is there every time the doors are open.

[lordmonar]

You can't develop a metric for a CAP squadron as no one has established any standards for what a squadron is supposed to be producing.

Personnel/personnel turn over/training metrics are all good....and are going in the right directions....but are useless with out "production" goals to base them on.

This is basic TQM and a prime example of why TQM failed in the USAF.   They wanted us to produce and report metrics....but with out production goals they were useless.  (and as a Comm guy we didn't have a production goals.....we had up time rate.......24 hours/7days a week.....)

[Eclipse]

Certainly a 10-year member with Level III, 2 ES ratings, active participation, and multiple PD ratings is more valuable to the organization
then someone with 20 years in who still has gold bars and shows up twice a year.

I would say certainly a 3 moth member whit level I ACTIVE PARTICPATION is more valuable then anyone else with all the PD/ES/ST ratings who only shows up twice a year.

Well, that's where it gets nuanced.

No one with 3 months in can do much of anything of real value to CAP.  With the exception of the random member who joins and hits NESA, an encampment, or some other major training
activity that just happens to occur within weeks or months of joining, most members in the first 6 months to a year are spending their timing finding the bathrooms and otherwise figuring out
what CAP is (or isn't).

Now, the value of an experienced member with a bunch of active ratings who can only show up a few times a year, will depend on when and why he shows up
and for how long, etc.  If he's not a PITA the rest of the year, maintains currency, and generally stays under the radar, but comes in and flies 100 hours for a flight
academy, or provides major training, it's going to be quite a few years before the new guys is equal in "value".

If, during his downtime, he's a constant PITA, off currency, and generally a nuisance when he shows up to an SLS to "start telling everyone how CAP works...", different
equation altogether.

[Eclipse]

You can't develop a metric for a CAP squadron as no one has established any standards for what a squadron is supposed to be producing.

Of course you can.

Perhaps it won't be meaningful to other units, but you can always establish your own local metrics of success.

The total lack of any criteria beyond "self-actualization" is the reason CAP continues to struggle with viability.

Unless you have goals, mandates, and strategies, you can't hold people's feet to the fire for "failure".

You also never know when you're "done". 

The current SUI / CI situation is a great example.  It presupposes standard evaluations across the board, however
we all know that isn't remotely true.  Not only are the inspectors all over the map in regards to abilities and experience,
but the definitions of much of the inspection points are wildly subjective at best.

[lordmonar]

Well, that's where it gets nuanced.

No one with 3 months in can do much of anything of real value to CAP.  With the exception of the random member who joins and hits NESA, an encampment, or some other major training

BS!    And that is where we part ways on volunteerism.   Day one we are getting our Senior Members involved in the job.  Just pulling records for the admin officer, just helping the logistics officer hand out uniforms, or count boot,  Assiting the comm officer in reading the numbers of the radios.   Driving cadets to and from the pick up point.

No one has to have a butt load of training to be valuable contribution to the unit and the mission.

activity that just happens to occur within weeks or months of joining, most members in the first 6 months to a year are spending their timing finding the bathrooms and otherwise figuring out what CAP is (or isn't).

That's a mentoring problem.......and not one we have in our squadron.

Now, the value of an experienced member with a bunch of active ratings who can only show up a few times a year, will depend on when and why he shows up
and for how long, etc.  If he's not a PITA the rest of the year, maintains currency, and generally stays under the radar, but comes in and flies 100 hours for a flight
academy, or provides major training, it's going to be quite a few years before the new guys is equal in "value".

If, during his downtime, he's a constant PITA, off currency, and generally a nuisance when he shows up to an SLS to "start telling everyone how CAP works...", different
equation altogether.

Yes....just because a CAP member is as you would call them an "empty shirt" does not mean they are not valuable themselves.   If they show up and produce....then that's okay by me.

[lordmonar]

Of course you can.

Perhaps it won't be meaningful to other units, but you can always establish your own local metrics of success.

The total lack of any criteria beyond "self-actualization" is the reason CAP continues to struggle with viability.

Unless you have goals, mandates, and strategies, you can't hold people's feet to the fire for "failure".

You also never know when you're "done". 

The current SUI / CI situation is a great example.  It presupposes standard evaluations across the board, however
we all know that isn't remotely true.  Not only are the inspectors all over the map in regards to abilities and experience,
but the definitions of much of the inspection points are wildly subjective at best.

I think we agree here.....except in maybe semantics....but yes....squadron X cannot know how well they are fulfilling CAP's or the Wing's mission goals.....with out having those goals.

[JeffDG]

Of course you can.

Perhaps it won't be meaningful to other units, but you can always establish your own local metrics of success.

The total lack of any criteria beyond "self-actualization" is the reason CAP continues to struggle with viability.

Unless you have goals, mandates, and strategies, you can't hold people's feet to the fire for "failure".

You also never know when you're "done". 

The current SUI / CI situation is a great example.  It presupposes standard evaluations across the board, however
we all know that isn't remotely true.  Not only are the inspectors all over the map in regards to abilities and experience,
but the definitions of much of the inspection points are wildly subjective at best.

I think we agree here.....except in maybe semantics....but yes....squadron X cannot know how well they are fulfilling CAP's or the Wing's mission goals.....with out having those goals.

Well, the good news here is that the person asking for assistance is a Group Commander...

That said, goals need to be set forth, then you can determine what Key Performance Indicators (KPIs) can be used to measure progress towards that goal.

To the OP,. sorry, I've not had a chance to really dive into those spreadsheets you sent me...probably won't be able to until the CI is behind me...

[Eclipse]

Yes....just because a CAP member is as you would call them an "empty shirt" does not mean they are not valuable themselves.   If they show up and produce....then that's okay by me.

Me too.  Empty shirts don't do that.

[bflynn]

Has anyone developed a comprehensive suite of metrics to monitor the health of (a) squadron(s)?

I trying to determine what the critical information is to monitor all aspects of a CAP unit.

If you would care to share, I would be interested in seeing it.

The question that hasn't been asked - are you sure you really want that?  I ask that because I've seen many, many organization ruined by bad metrics.  I'm in the process of writing a white paper for work on metrics and using them to drive business change.  But part of my white paper is NOT using them in certain circumstances.

Once you implement metrics, people will begin to conform to them.  Then people will begin to game the metrics to make themselves look better.  If the metrics are bad then pretty soon, they are chasing the metrics and not doing what needs to be done to make a strong squadron.

My advice here is not to have a comprehensive set of metrics - just have a good set.  Track simple things that you know are good for the squadron - number of members, number of hours in air/ground training, etc.  Maybe track money spent.  Avoid complex metrics such as $ per hour per member because what that means becomes twisted and is easier to manipulate.  You can't game number of member, either you recruit and get more or you don't and have less.

Last - I'd say that it's fine if you want to include some metrics as indicators of personnel performance, but keep it light.  You do not want people trying to game any systems and you don't want metrics to be demotivating...they can be.  So, I'd say never compare people to others using the metrics.  There can be many, many reasons outside the metrics on why someone isn't "performing".  For example, I think its easy to say that the squadron in Oakridge Or is going to have a whole lot more trouble recruiting than say Portland will.  Or even than the previous commander in Oakridge had.

Metrics can be a great TRENDING tool, if used that way.

[RiverAux]

WIWASC I kept track of overall member retention, senior retention, cadet retention and did the same for retention of 1st year members. 
Those were the most useful to me. 

You could really go crazy tracking just about everything under the sun but frankly no one other than you is going to care about them.

Several years ago they implemented a "Commanders Dashboard" at the wing level that kept track of certain things but I'm not sure how much it has been utilized as a tool. 

[Eclipse]

Several years ago they implemented a "Commanders Dashboard" at the wing level that kept track of certain things but I'm not sure how much it has been utilized as a tool.

The numbers are so high-level as to be mostly useless.

[Walkman]

...I ask that because I've seen many, many organization ruined by bad metrics. 

This makes a lot of sense to me. People can have the tendency to focus on the wrong thing, like chasing a number, and lose sight of the bigger picture. It reminds me of the airline crash in the 80's in Florida. On approach the landing gear light didn't light up to show the gear was down, while they kept fiddling with it and trying to figure out if the landing gear was working they forgot to fly the plan and crashed, killing lots of people. Investigation later revealed the the light had burned out.

[Eclipse]

And I've seen more then a few fixed with good metrics, so, use the good ones.

Regardless, without them, there is nothing but status quo.