Rand Survey forthcoming

Started by RogueLeader, November 10, 2016, 01:43:37 AM

0 Members and 1 Guest are viewing this topic.

RogueLeader

I just got an email forwarded to me stating that on 01 December 2016, NHQ, via the Rand Corp. is sending out a Survey about Encampment.  See the attached memo.
WYWG DP

GRW 3340

Eclipse

It might be nice if the results of these were released at some point.

"That Others May Zoom"

ProdigalJim

Speaking of surveys, did the results ever come out from the membership survey this past Spring?
Jim Mathews, Lt. Col., CAP
VAWG/XP
My Mitchell Has Four Digits...

CAPDCCMOM

#3
I have only been in CAP for a very short time, but in that time, I have seen several of these Surveys come out. Each time they are released, we are assured that the information being gathered is to benefit our membership as a whole, and deal with dismal Recruiting and Retention numbers. What I have not seen, are the results of the surveys. I have not heard any discussion on what is being done to correct glaringly obvious issues. The survey is filled out, submitted, and then we are thanked for our valuable service. Then CRICKETS!

We have new Senior Members come in and one of two things happen, either they have to sit and wait for six months until the Squadron Leadership has time to have the discussion portion and sign off, or they are railroaded so quickly through level one that they don't learn anything. Then they get put into a Higher level job than they should be, just because there is a critical vacancy and an SUI coming up. Cadets come into the Program, parents slow down to 50 mph and push them out the door. Cadet gets frustrated because they learn that they have to WORK for promotions, we do not run an EGAT program.

Should you, as a Senior Member, survive your first six months you get to learn all about Wing politics and the "Good Ol' Boys/Girls Network". You get to see the military wannabes in all of their cosplay glory. You go to a Wing Conference and see people that make the Blues uniform look like a blue package of Jimmy Dean sausage. If you are lucky, you get to see members of Wing Leadership get blind drunk when there are Cadets about 200 yards away. You have watched, and have seen the Core Values, that you believe in ripped to shreds.

This is happening all of the time, all over the Country, but by all means let's fill out another survey. That should fix everything. Meanwhile, how many people are looking at their re-new notice on e-services and rather than saying, "yes, I need to take care of that", they are saying, "Glory Be, just X days".

Spam

Quote from: CAPDCCMOM on November 10, 2016, 07:35:36 PM
I have only been in CAP for a very short time, but in that time, I have seen several of these Surveys come out. Each time they are released, we are assured that the information being gathered is to benefit our membership as a whole, and deal with dismal Recruiting and Retention numbers. What I have not seen, are the results of the surveys. I have not heard any discussion on what is being done to correct glaringly obvious issues. The survey is filled out, submitted, and then we are thanked for our valuable service. Then CRICKETS!

We have new Senior Members come in and one of two things happen, either they have to sit and wait for six months until the Squadron Leadership has time to have the discussion portion and sign off, or they are railroaded so quickly through level one that they don't learn anything. Then they get put into a Higher level job than they should be, just because there is a critical vacancy and an SUI coming up. Cadets come into the Program, parents slow down to 50 mph and push them out the door. Cadet gets frustrated because they learn that they have to WORK for promotions, we do not run an EGAT program.

Should you, as a Senior Member, survive your first six months you get to learn all about Wing politics and the "Good Ol' Boys/Girls Network". You get to see the military wannabes in all of their cosplay glory. You go to a Wing Conference and see people that make the Blues uniform look like a blue package of Jimmy Dean sausage. If you are lucky, you get to see members of Wing Leadership get blind drunk when there are Cadets about 200 yards away. You have watched, and have seen the Core Values, that you believe in ripped to shreds.

This is happening all of the time, all over the Country, but by all means let's fill out another survey. That should fix everything. Meanwhile, how many people are looking at their re-new notice on e-services and rather than saying, "yes, I need to take care of that", they are saying, "Glory Be, just X days".


I don't generally encourage nor allow my cadets to go to Wing Conferences because of much of what you say (which I've seen in the five Wings I've served in). When cadets are asked to help carry a keg to their commanders room, and officers are still visibly inebriated at general assembly at 0800... that's it for me, I'm not approving it as an activity for my cadets. There was a time when I very controversially refused to sign IACE applications for the same reason, after two of my C/LTCs came back talking about getting "legally Da-Runk" overseas with their escorts, but that hopefully is a thing of the 90s/00s now.


It was partly because some of us put our foot down and started bashing the drunkenness and carousing that IACE cleaned up their act (also after a rather expensive bill from one incident in Germany). One would hope - hope! - that the survey would elicit open ended comments such as you offer to try to force change, but I don't see it being more than five point scales and "rather than" pick lists again from a World Renowned Research Lab.


I've made the point (as a former DCP) about drinking at Conferences, and the inconsistency with the so called DDR message. It continues. I've moved on back to a local CO job where I can be the change I want to see. If whatever our varied points are cant make it through a nice sterile five point Likert rating scale... NHQ wont see or care and it wont get addressed at National, Region, or Wing level through regulatory or program changes, nor enforced at local level.


V/r
Spam





Ned

Quote from: CAPDCCMOM on November 10, 2016, 07:35:36 PM
I have only been in CAP for a very short time, but in that time, I have seen several of these Surveys come out. Each time they are released, we are assured that the information being gathered is to benefit our membership as a whole, and deal with dismal Recruiting and Retention numbers. What I have not seen, are the results of the surveys. I have not heard any discussion on what is being done to correct glaringly obvious issues. The survey is filled out, submitted, and then we are thanked for our valuable service. Then CRICKETS!

I think there may be a misunderstanding here, and if my shop has contributed to it, I sincerely apologize.

As my colleague Curt LaFond mentioned in the memo linked above, the survey announced in the memo is not a "membership-wide" Recruiting and Retention-related survey.  This is a specific CP-only survey that focuses solely on our terrific encampment program and its synergistic effect on our cadet program.  This is the second such survey, and is a performed by an outside group - the Rand Corporation - which has been tasked with this particular survey by our Air Force colleagues.  As you know, the AF has generously given us over a million dollars over the past two years to help fund cadet attendance at encampment.  This survey is designed to document whether or not they (and the taxpayers) are getting their money's worth by measure attitude and participation changes that can be attributed to encampment participation.

The survey, of course, is completely voluntary for our cadets and their parents.  We specifically promise confidentiality for participants in order to encourage candid and complete responses.  Accordingly, the data is only released to us at NHQ in the aggregate and we use it to plan for further improvement of the encampment program.

I'm sure Lt Col Ninness and his crew could respond more completely about membership-wide command climate type surveys, but that is not in our lane. 

Thank you for your work with our cadets.  You make a difference.

Ned Lee
Col, CAP
National Cadet Program Manager.




Spam

Hi, Ned.

I do appreciate your openness and willingness to seek inputs (thank you!), and Curt is in my experience similarly responsive.  Your ability to get feedback from such a survey device is necessarily limited, however, by the structure of the tool. There's a risk that you might see metrics (subjectively based, but numeric indices) rather than feedback expressed with ideas and constructive (hopefully!) criticism.


Will the survey this year have a free text field (or fields, for specific areas) designed to elicit open ended comments such as you see here? If so, and if it prompts for constructive comments vice mere complaints, you might have an outstanding tool there.


Thanks,
Spam


(PS, before someone goes ad Hominem... the ability to be responsive and civil doesn't necessarily correlate to mutual agreement on all topics, nor should we expect same).


Ned

Quote from: Spam on November 11, 2016, 03:18:06 PM

I've made the point (as a former DCP) about drinking at Conferences, and the inconsistency with the so called DDR message. It continues. I've moved on back to a local CO job where I can be the change I want to see. If whatever our varied points are cant make it through a nice sterile five point Likert rating scale... NHQ wont see or care and it wont get addressed at National, Region, or Wing level through regulatory or program changes, nor enforced at local level.

Jeff,

I'm not sure some of this is fair to the corporate CP staffers.

Initially, the "DDR Message" is that drugs (including alcohol) are never appropriate for cadets, and drug use has countless negative consequences for youth.  We offer education, alternate activities, and community outreach in our DDR Program.  (See CAPR 52-22.)  We do not teach that responsible drinking by adults is improper or immoral. 

CAP has no policy that governs drinking by CAP seniors, per se.  Unless at a cadet activity, seniors are free to have a glass of wine in appropriate circumstances. 

Obviously, there can be a CPP aspect in some circumstances, which is why I wrote and included language in the CAPR 52-16 that specifically restricts drinking by senior members at cadet activities.

The great majority of conferences are conducted without alcohol-related incidents.  Of course, humans are humans, and sometimes an adult might have a drink too many.  I have attended literally hundreds of wing, region, and national conferences in my time in CAP, and have seen a few incidents.  It sounds like you and others have seen such things, too.  In those circumstances, we rely on local leadership to take the appropriate actions to safeguard the members involved.  It is unrealistic to think that NHQ could (or should) become the "CAP banquet police."

On occasion, CP has considered proposing a simple, bright line rule that seniors cannot drink or smoke in the presence of cadets.  Although reasonable minds can differ about the value of such a rule in helping cadets maintain a drug free lifestyle (there isn't a lot of research that supports that it would result in a measurable decline in smoking or drinking rates), the consensus is that such a rule would simply result in cadets being excluded from senior-focused activities such as ES training missions, actual missions, and things like wing conferences.  And that is, in our view, problematic.




Ned

Quote from: Spam on November 11, 2016, 06:34:58 PM
Your ability to get feedback from such a survey device is necessarily limited, however, by the structure of the tool.

See, that's the thing.  We (NHQ) didn't design the instrument.  It was designed by the PHD-level Deep Thinkers at the Rand Corporation who have devoted their entire professional and academic careers to designing survey instruments and analyzing the results.  And - to be fair - we are not the primary customers of this particular survey, it was commissioned and funded by the AF to ensure that their CEAP money is well spent.  We are, of course, happy stakeholders and anxious to use the aggregate data to further improve our programs.

We DID offer feedback after last year's survey (much of it gathered right here on CT), and work with a Rand representative as the survey is developed.  But the bottom line is that the survey is designed by professionals, and we are not the primary customer.  We just want to keep the primary customer very, very happy so they keep allocating hundreds of thousands of dollars for the CEAP program.

Which, by the way, appears to be working to the AF's satisfaction.  In coordination with the AF budget programmers, it appears that CEAP is here to stay for several years at least.  Yay!!

QuoteWill the survey this year have a free text field (or fields, for specific areas) designed to elicit open ended comments such as you see here? If so, and if it prompts for constructive comments vice mere complaints, you might have an outstanding tool there.

Let me double check, but I don't think the majority of the questions allow for open-ended comments.  I suspect because open ended natural language comments are difficult to code and quantify.  But let me check and get back to you.


Spam

Ned, those are great responses, thanks.

First, I'd like to delete the words "or care" from my earlier post... my intent was to convey that NHQ staff might never see the feedback, not that you'd not care. Sorry about that.


Next, I'm grinning because I've worked with Rand before and my current shop performs similar analyses for DoD customers (we're also filled with piled higher and deepers and more of the sames like me). I appreciate the situation regarding the tools construction, as I've had difficulties gathering open ended comments from DoD aircrew and UAV operators. I can imagine the problems with a varied, far larger respondent pool. I hope they could help gather subjective commentary for you but - and you need to consider this as a program manager - would you really want unfiltered data (e.g. possibly actionable complaints and personal attacks) going to your USAF customer. I hadn't considered that angle, frankly, and it might not be a good idea from that aspect.


Finally, on your consideration of the bright line rule, I would consider that any activity with cadets present is a de facto cadet event.  In terms of impact on cadets in ES and on missions - how many of our ES staff really get bombed at a mission? I've not seen one in ~35 years in CAP ES, and would doubt that its a factor.

The main impact, in my experience, is at conferences and on occasion at encampments. Were we an  organization without minor youth members there would be no issue but, there are three: impaired supervisors standing in loco parentis, ORM issues of safe behavior, and the leadership issue of setting a standard of behavior in front of cadets.

Anecdotal: at a 1991 encampment I was Deputy for, I was asked by the CO to reprimand four officers who went off base drinking (in Class As) and returned to a common barracks at 0300 - and who were driving both on base while drunk and were driving vehicles with cadets at 0800 before we got the word on them, as no one wanted to snitch. All three factors, there. I know of similar examples recently, sad to say, so I was glad to see your 52-16 updates, which give some teeth (if enforced).

Anecdotal: We just heard WED night from a former cadet about the carousing at conferences and what a poor influence it had on him and his peers (within the last 5 years) which mirrored our similar observations over the years/decades. I don't think it would be unreasonable at all to draw that Bright Line, and segregate cadets at a Cadet Conference that is on a separate date to avoid the frat boy effect intended/actively planned for at Wing Conferences as we do with SLS/CLC/TLCs, all held without cadets.


Interesting discussion.

V/r
Spam






RogueLeader

Quote from: CAPDCCMOM on November 10, 2016, 07:35:36 PM
I have only been in CAP for a very short time, but in that time, I have seen several of these Surveys come out. Each time they are released, we are assured that the information being gathered is to benefit our membership as a whole, and deal with dismal Recruiting and Retention numbers. What I have not seen, are the results of the surveys. I have not heard any discussion on what is being done to correct glaringly obvious issues. The survey is filled out, submitted, and then we are thanked for our valuable service. Then CRICKETS!

We have new Senior Members come in and one of two things happen, either they have to sit and wait for six months until the Squadron Leadership has time to have the discussion portion and sign off, or they are railroaded so quickly through level one that they don't learn anything. Then they get put into a Higher level job than they should be, just because there is a critical vacancy and an SUI coming up. Cadets come into the Program, parents slow down to 50 mph and push them out the door. Cadet gets frustrated because they learn that they have to WORK for promotions, we do not run an EGAT program.

Should you, as a Senior Member, survive your first six months you get to learn all about Wing politics and the "Good Ol' Boys/Girls Network". You get to see the military wannabes in all of their cosplay glory. You go to a Wing Conference and see people that make the Blues uniform look like a blue package of Jimmy Dean sausage. If you are lucky, you get to see members of Wing Leadership get blind drunk when there are Cadets about 200 yards away. You have watched, and have seen the Core Values, that you believe in ripped to shreds.

This is happening all of the time, all over the Country, but by all means let's fill out another survey. That should fix everything. Meanwhile, how many people are looking at their re-new notice on e-services and rather than saying, "yes, I need to take care of that", they are saying, "Glory Be, just X days".


First, I want to say thank you for your comments, seriously, thank you.  I know that after 12 years in the program, it can be very easy to loose perspective as to how new people see the program, so a new(er) members view is important to  keep in mind on how the process works.  Again, thank you.

Second, again in all seriousness, what do you expect NHQ to DO about these issues that you bring up?  I agree with you that they are issues that need corrected.  Do you expect them to make site visits?  Do you expect them to publish more regulations on specific timelines to achieve "optimal" efficiency?

I totally agree with you that something needs to be done, so let me ask you a question:  What have YOU done to correct the issue?  Have you spoken up to the offending member?  Have you brought it up their commander?  Have you made mention of it the the staff officer that is involved in the Office of Primary Responsibility?  Have you applied for the position to be able to correct the issue?

Here in my wing, pilots are an issue with how they train for ES.  I applied for, and got, the Assistant Director of ES.  Despite the improvement in some areas, pilots are still being pilots and not getting with the program.  The GOB network is still strong, although some inroads have been made.  A while back, I was invited to apply for a position that would allow me to make the changes that I, and many others, feel that need to be made.  I submitted the application.  Even if I don't get it, I know that I'm doing everything I can to influence those around me to make better choices, and to do the right thing.

Everybody has three choices:  They can work to correct the issues, they can give up on trying to make the change they want to see, or they can walk away.

Be the change you want to see.  At the end of the day, it comes down the the choices of one, yours.
WYWG DP

GRW 3340

ProdigalJim

Quote from: RogueLeader on November 15, 2016, 06:01:57 PM
Quote from: CAPDCCMOM on November 10, 2016, 07:35:36 PM
I have only been in CAP for a very short time, but in that time, I have seen several of these Surveys come out. Each time they are released, we are assured that the information being gathered is to benefit our membership as a whole, and deal with dismal Recruiting and Retention numbers. What I have not seen, are the results of the surveys. I have not heard any discussion on what is being done to correct glaringly obvious issues. The survey is filled out, submitted, and then we are thanked for our valuable service. Then CRICKETS!

We have new Senior Members come in and one of two things happen, either they have to sit and wait for six months until the Squadron Leadership has time to have the discussion portion and sign off, or they are railroaded so quickly through level one that they don't learn anything. Then they get put into a Higher level job than they should be, just because there is a critical vacancy and an SUI coming up. Cadets come into the Program, parents slow down to 50 mph and push them out the door. Cadet gets frustrated because they learn that they have to WORK for promotions, we do not run an EGAT program.

Should you, as a Senior Member, survive your first six months you get to learn all about Wing politics and the "Good Ol' Boys/Girls Network". You get to see the military wannabes in all of their cosplay glory. You go to a Wing Conference and see people that make the Blues uniform look like a blue package of Jimmy Dean sausage. If you are lucky, you get to see members of Wing Leadership get blind drunk when there are Cadets about 200 yards away. You have watched, and have seen the Core Values, that you believe in ripped to shreds.

This is happening all of the time, all over the Country, but by all means let's fill out another survey. That should fix everything. Meanwhile, how many people are looking at their re-new notice on e-services and rather than saying, "yes, I need to take care of that", they are saying, "Glory Be, just X days".


First, I want to say thank you for your comments, seriously, thank you.  I know that after 12 years in the program, it can be very easy to loose perspective as to how new people see the program, so a new(er) members view is important to  keep in mind on how the process works.  Again, thank you.

Second, again in all seriousness, what do you expect NHQ to DO about these issues that you bring up?  I agree with you that they are issues that need corrected.  Do you expect them to make site visits?  Do you expect them to publish more regulations on specific timelines to achieve "optimal" efficiency?

I totally agree with you that something needs to be done, so let me ask you a question:  What have YOU done to correct the issue?  Have you spoken up to the offending member?  Have you brought it up their commander?  Have you made mention of it the the staff officer that is involved in the Office of Primary Responsibility?  Have you applied for the position to be able to correct the issue?

Here in my wing, pilots are an issue with how they train for ES.  I applied for, and got, the Assistant Director of ES.  Despite the improvement in some areas, pilots are still being pilots and not getting with the program.  The GOB network is still strong, although some inroads have been made.  A while back, I was invited to apply for a position that would allow me to make the changes that I, and many others, feel that need to be made.  I submitted the application.  Even if I don't get it, I know that I'm doing everything I can to influence those around me to make better choices, and to do the right thing.

Everybody has three choices:  They can work to correct the issues, they can give up on trying to make the change they want to see, or they can walk away.

Be the change you want to see.  At the end of the day, it comes down the the choices of one, yours.

^^^^^ Couldn't agree more. It's the old thing about being part of the solution rather than part of the problem.

And yet...

Sometimes we make it institutionally difficult for folks to be part of the solution. And there are some things that really *need* to happen at a national scale if they're to have the desired effect.

At RSC this year, my seminar produced a white paper on recruiting and retention. We surveyed the available research on retention in volunteer-membership organizations, and also looked at CAP's existing practices to see where they lined up. Overall we found what I think were some pretty interesting things. Essentially, our group contended that CAP already has Best Practices in place, but executes them unevenly and has under-resourced the areas which have been shown to be crucial to volunteer satisfaction and retention in other member-volunteer organizations.

Of nine Best Practices identified in one of the studies we looked at, CAP can be fairly described as having adopted ALL of them to some degree.
   
1. Regular supervision and communication with volunteers
2. Liability coverage or insurance protection for volunteers
3. Regular collection of information on volunteer numbers and hours
4. Screening procedures to identify suitable volunteers
5. Written policies and job descriptions for volunteer involvement
6. Recognition activities, such as award ceremonies, for volunteers
7. Annual measurement of the impacts of volunteers
8. Training and professional development opportunities for volunteers
9. Training for paid staff in working with volunteers

Three practices in particular were found to have a high degree of correlation to member retention, and these practices already figure prominently in CAP management and activities. They are 1) hosting recognition activities for volunteers, 2) offering training and professional development opportunities for volunteers, and 3) screening for suitable volunteers and using that screening to match volunteers to appropriate opportunities. "These volunteer management practices all center on making the experience worthwhile for the volunteer," the study authors wrote. "Retention appears to be very much a product of what charities do directly for their volunteers."

We also looked at another separate study that examined nonprofits and charities with good retention rates for volunteers carrying out their organizations' missions, which found that as a benchmark appropriate support infrastructure can be expected to cost about $300 per volunteer in 2002 dollars (when the study was published). Not accounting for inflation and based on the current number of adult members, that would suggest roughly $9.8 million should be spent annually on professional personnel and training support infrastructures for members executing the CAP mission. The FY 16 appropriated plan calls for roughly $1.3 million to be spent on professional services and professional development combined, or only about $38 per member. This obviously does not account for the value of the services and support contributed by volunteer leaders (instructors and staff at RSC, for example) which, fairly counted, might well take that per-member figure much higher. Even so, the wide gap at least merits further study to understand whether enough continuous training support is getting to members in the field and whether those member-level training and support functions may be under-resourced.

Themes that emerged from all three studies our seminar group used suggest that perhaps larger numbers of CAP members are becoming dissatisfied with the perceived cost/benefit ratio of membership. What we said in our RSC paper was, "Put another way, CAP may not need to adopt a host of new practices but instead may need to work harder to deliver on the promises already made via existing practices."

If recognition, training, professional development and upfront screening/matching and continuous management and professional support really are crucial – and the studies show those practices correlate strongly with retention for large-scale nonprofits – CAP should not ask whether it is or is not doing these things but instead should critically and candidly examine whether it is doing them well.

Does CAP do a good job of recognizing volunteers consistently through awards, decorations and advancement, or are we still inconsistent in executing the details of processes for awards, decorations, ratings and recognition? How diligently do Squadron CCs pursue writing Form 120s? How often do Group, Wing and Region staffs use the process to recognize subordinate commanders, or high performers whose work they know personally?

While we offer training and professional development opportunities, are they of a high-enough quality to be recognized as a valuable member benefit? Although the studies suggest that an early, upfront emphasis on acculturation, on-boarding and equipping new volunteers aids in retention, CAP instead relies heavily on self-study and computer-based training during the crucial early phases. In our zeal to reduce cost and increase convenience in training, have we diminished the professional value of training? Do our PD materials really reflect the level of professionalism we need in an "officer corps" preparing to take on more responsibilities as part of the Total Force?

Has our new-member screening process become too reactive and defensive, designed to protect the organization without enough regard to creating a pipeline that matches prospective members' skills, abilities and interests with opportunities in CAP?

Do complaints about burdensome paperwork, computer systems and the like mask an overall dissatisfaction with the level of professional support to the field (perhaps stemming from under-resourcing professional support and professional development well below even the 2002 benchmark)?

In our RSC paper we were careful to say that "we pose these questions and ideas not because we have solid evidence that these are driving factors. That would be impossible in the course of a few days' work by an RSC Seminar. But as sitting or graduated commanders, seasoned and in the trenches, drawn from six separate Wings, we have observed enough of this at the Squadron and Group level to lead us to believe that examining these questions more closely would be fruitful."

Sitting here seven months later, nothing has changed my mind that this is still true.

If anyone's interested in either our paper or the references, I'm happy to offer them up.



Jim Mathews, Lt. Col., CAP
VAWG/XP
My Mitchell Has Four Digits...

CAPDCCMOM

Rouge, Thank you you for your thoughts.

In fact I have worked at being the change that I want to see, at my own Squadron level. When ever we have Cadets, or will be in contact with Cadets, there is NO alcohol consumed, end of discussion. Since I have completed my own Level 1, we have worked to ensure that our new Senior Members get the mentors that they need, and are not left waiting for six months. Within my Squadron we adhere to 39-1, with out any weigh ins or interference. We are lucky we still uphold Integrity.

Regulations need to be upheld and enforced consistently, that seems to be the biggest issue that I can see. Forgive me, but I fail to see the value of surveys, if nothing constructive is done with the data, and if the information is not dispersed to where changes can be made. Have i tried to apply at the Wing level? Yes, but the GOB/G network is hard at it. So, I am the change that I want to see in my own corner of CAP.