eServices problem?

[Nuke52]

The "as is" with eServices shows a disturbing lack of vision, and consideration for members.  I appreciate the example (shoving changes straight into prod), but, respectfully, it's like telling me having the flu ain't so bad, because someone else has pneumonia.

Rant complete. And pointless, like as not.

How strange.  Anonymous ranting on the internet is usually so productive.  Maybe you're not doing it right.

Perhaps he needs more sarcastic replies to get it working better.  At the very least, it would keep him motivated in his desire to improve the organization...

[Eclipse]

And NHQ is severly resource-constrained.  There have been multiple rounds of staff reductions in recent years.  Mr. Rowland and his team have done terrific work trying to balance the missions and needs of the HQ.  I don't envy the resource allocation choices he has had to make.

An interesting thing to say, since the needs of the mission are supposed to >be< the needs of HQ, and the mission is supposed to come first.

But the bottom line is that to achieve the kind of modern eServices that you and other members would like will almost certainly take hundreds of thousands of dollars to develop and code a new system.  (Even in Sharepoint.)

Which obviously we don't have.  It would take a substantial dues increase to raise that kind of funds, which I suspect the membership would find unappealing.  Alternatively, perhaps you can identify which major programs should be eliminated to provide additional support to IT.

I don't buy that, not even a little.  Would a typical, government-eque, big-iron RFP, etc., etc., system require hundreds of thousands?  Probably.

Would what we need?  No.  In fact, I'd go so far as to say that if this project actually "interested" any hardware or software vendor, then it's not headed in the right direction.

This isn't the '90's and there are plenty of ways to get robust services inexpensively or even free, and I would hazard a guess that a competent single programmer
working in an open source, robust and free application could bolt the basic thing together in a weekend, maybe a couple if a new season of "Game of Thrones" has been released.  The system itself
is pretty "simple", has pretty basic rules, and is not really dynamic in the ways most internet ecommerce systems are today.  My wing has guys that do this stuff for "fun" (I know they need help),
tossing together systems like it was nothing, and usually the only roadblocks they encounter are NHQ's closed systems, lack of an API or documentation, and constant
changes with no thought to what's been done in the field.

We have literally hundreds, if not thousands of IT folks dying to take a crack at eServices, on their own time, for free.  No one has ever asked.

I'll add another week for a UI team, and $30 for a whiteboard, since it does not appear that much time has ever been spent on UX design.

After that, it's just a matter of scale and bandwidth.

And it should >not< be hosted under anyone's desk at Maxwell, or inside any other CAP building or facility.
I'd add a Venti to the bet that a couple of phone calls could get us free hosting somewhere for the "cost" of
an endorsement and a little free press.

Like any disruptive change, it's not "easy", but that doesn't mean it isn't "simple".  The biggest part is getting past "not invented here".

[SunDog]

It's great to hear so many good ideas, so I'm kinda glad I stirred it up a bit. Someone contacted me offline, said there are only two (2) people in NHQ IT. Wow, bless 'em both! Do they even get days off?

Ned, I haven't seen CAP's budget, and I'm not qualified to suggest a place to cut, in order to generate bucks to fix eServices; maybe buy one less glass 182 next year? If that money comes out of a "diffrent bucket", I'd say rearrange the buckets - Senior Leadership go to bat for membership, get the money moved, right?  Delay some capital expenditures, accelerate some asset sales, and get changed whatever needs to be changed to use the money for what's broken. If that won't cover it, look at some of the ideas already posted here?

I am qualified to evaluate software. And remediation, and there are sound ideas posted here, worthy of consideration by Senior Leadership. It 's kinda cheered me up a bit, to see smart folks with good ideas take a swing at this. Geez, it doesn't have to be a huge, monolthic, all-or-nothing, 1990's ERP type monstrosity project. Like someone said, break it down, map it out, use the membership.

[coudano]

Look, here's something that EVERYONE in CAP can do to help out the online system.
Test it.

When you find something "wrong", write up a detailed report for the IT people to look at.
They really can't use "hey this thing sucks"  or  "this this is broken"
However, they CAN use:
1.  a detailed description of what you were trying to do (i'm trying to add promotion data for a prior achievement to capid 123456)
2.  a detailed description of the steps you took leading up to the error
3.  the specific error, copied and pasted, or screenshot from your screen (so they can replicate it)
4.  it never hurts to throw in a suggestion for a fix, if you have that up your sleeve
5.  your operating system, browser+version

I did some pretty extensive (mostly unsolicited) software testing, and reporting for the current rendition of ops quals, most of the cadet programs modules, and online testing.  I found that thorough reports were well received, and especially with regards to the cadet programs modules /quickly/ implemented.  A few times I interacted with the IT shop at nhq (usually replying to a report with further questions) they were prompt and polite.  --I will say that is an improvement from the experience several years before, so it does seem to be getting better.  If there was a ribbon for every bug or suggestion reported and implemented, i'd have a ribbon full of silver clasps.


I tend to find the airplane stuff pretty silly as well.  And I am also no big fan of WMIRS for mission paperwork, although I haven't actually used it in a while (is it still the current application of choice?).  I remember it striking me as something I may have written myself, in my free time (in other words, not very good).


Programming is tricky business.  There is no one "right" way to solve a problem.  No matter which way you choose, someone else will make (probably valid) arguments that the other way would have been better.  No matter which features you implement, someone will make (probably valid) arguments for the features that you didn't.  No matter how many things you fix, someone will always be quick to point out the things that are still broken.  That's just life.  --In CAP, additionally, the "rules" change about every 6 months, so what you produce that works correctly today is invalid and "broken" tomorrow.  One thing that CAP doesn't do very well is 'splain "hey I (the undersigned) am the program manager in charge of this, here is what I have chosen and why, and that lines up with our (published) policies in the following ways."


Personally, if we were going to burn nhq's IT to the ground and re-build it,
My preference would be to focus our staff on data management.  Make sure that we are capturing the data in the way that we need to capture it, and make sure it is highly available, highly accessible, backed up, snapshots going back into history, and properly secured (per field, per user).  Then provide API to the membership to read and write that data.  That would allow the field to write their own apps, and ideally share them amongst units.  Yes, there would instantly be 1,000 apps to pull up a squadron roster.  But the best ones would sort of float to the top quickly.  When the 'business rules' become open source (within the org), anyone in the community can review them all, and point out dependency failures or conflicts, and propose a resolution.


The biggest obstacle, I think, is almost certainly paradigm shift about data ownership and protection.
And i'm not really sure as I don't have a current "feel" for the situation, but in the past I have recalled a sort of 'palpable' sense of "us vs them" between the nhq staff, and CAP membership.  Particularly when it comes to ownership and control of things, in particular, as IT.  To that end, I think there should be a national IT 'committee' made up of volunteers.  And CAP's "CIO" should be a member at large.  This group would be responsible for clarifying a member-based, unit-based vision, mission, purpose, and methodology for CAP's IT structure.  The paid staff (full time and contractors) would then execute that on a M-F 8-5 basis.

I am nearly a master rated IT in CAP (and it's my civilian job, at which i'm reasonably successful), and I can't tell you off the top of my head who CAP's chief information officer is, or name one thing I have seen that person (or their office) publicly do, ever.  If you want to know why the situation is what it is in CAP, start there.  And whoever that is, might just have their hands tied from upstairs, for all I know...  It's not an enviable position in the current climate.


Darin can tell you stories of olde about proposed (and shot down) content management structure, to put every squadron (group, wing, and region) in CAP under a common web service, with a standard "look and feel", with details for each unit to manage.  I _STILL_ think that's a GREAT idea.  If a unit wants to go "above and beyond" they can make another site and link to it from their 'official' cap one.  But those units who can't spell internet, are still going to have something presentable, and as accurate as possible.  // This idea should be implemented in conjunction with the above.  You could use CAP-API modules in your "official" squadron website.


The full time guys at NHQ, I imagine, in addition to their e-services duties, probably also have to jockey toner and copy repair, troubleshoot computer breakage, add and delete accounts to the nhq network, audit and inventory hardware and software, probably run the office firewall, and who knows what else.  And they SHOULD do that stuff; somebody has to.  Supposing for my little proposal, two more fulltime people, and we're talking about specialized database, sysadmin, and API programmers, here...  There's probably $200k right there, annually (at pretty much bargain basement range).

I don't think you WANT free hosting...  I think you want to pay for it, and I think you want to get what you paid for (including, frankly, SLA).  But I do think you want offsite hosting, probably in a HA cloud center.  That's probably not going to be cheaper than a box or 2 in the closet at NHQ.  HA data is also going to take a non-trivial amount of disk space, which is actually fairly cheap...  but not free.

60,000 members (records) across whatever, like 25 tables, that's like 1.5Mil records.
Which is enough to blow up a database server, if it's configured poorly (and from what i've seen of capwatch, ours is configured quite poorly)
--which is where the specialized db admin comes in
It is not an insurmountable amount of disk space we are talking about, though.  The expense comes in making it highly available, backing it up offsite regularly, and in adding snapshots for historical statistical analysis (this will balloon as when a member quits they were on yesterday's capwatch but they are not on today's).

The final piece of the picture is processing capability.  The 'report server' (probably going to wind up being crystal, right?) is going to have to be able to handle at least hundreds of 'interested' members, developing reports.  That many again, running reports.  That server is going to get locked up and brain dead, and require zombies to be killed, and so forth and so on.  "dumb" users are going to create reports that sap the system down.  The job queue is going to get kludged.  This is all just standard stuff for report engines, and it takes someone to restart the service; or the whole machine.  Maybe even multiple machines, load balanced...  That's just for apps that read the data.  Writing to the database would need to be much more closely controlled, and apps that did it would probably need to be thoroughly tested, if not professionally developed.


**The current system definitely needs to continue running as well, whilst something 'new' is stood up as a replacement.


Anyway, it's doable.
But I think ned has it about right at 'hundreds of thousands'.  Probably about $250k or so.  That's about $4 per year annual dues org wide.
The changes to the organizational structure and attitude, however, would cost nothing at all (well not in dollars anyway), and are quite frankly pre-requisite to any technical solution that may be attempted.  There is no point whatsoever in spending one penny on "improved capability" until those things are "fixed".

[SunDog]

"The changes to the organizational structure and attitude, however, would cost nothing at all (well not in dollars anyway), and are quite frankly pre-requisite to any technical solution that may be attempted.  There is no point whatsoever in spending one penny on "improved capability" until those things are "fixed".

Nicely said, clearly articulated. . .Real world, likely we'll need to wait on normal attrition/changes in management.  I think, reading between the lines, that eServices is perceived as "good enough to get by" by Senior Leadership.  Not intending to be cynical, but the impact of eServices' poor implementation on NHQ staff is a lot lighter than on the membership.  The pain isn't shared all that much.  I am pretty certain eServices (and WMIRS) hurts retention and mission readiness - I don't think anyone walks away just because these apps are awful, but they do figure into the sum of frustrations.  I think Senior Leadership can truthfully say both apps work - because they do; they just miss the bigger picture that they are a major frustration point for members. There are a LOT of volunteer member hours being sucked up by these apps. Those hours come out of availability for other tasks. TANSTAAFL, right?

But, speaking to the fix! Someone would have to manage the project, regardless of the mix of resources used.  The API route, with data hosted centrally, sounds good, and doable. I imagine there is a retired, solid, experienced PM out in CAP Membership land, who might spare a year to shepard this.  IF IF IF NHQ would allow it to proceed. As you say, pointless to try without NHQ buying in. And if they sustain a "NIH" attitude, might as well stay home.  Maybe someone on this thread who HASN'T made 'em mad could make the pitch? It would be low impact ($$$ and time) on NHQ IT to get started - hand over the schema and rules, and answer some questions. . .if the design is a success, the hosting and provisioning for "production" could get worked out late in the game. Heck I'd chip in $10 a year for a couple of years. . .

[Eclipse]

I think, reading between the lines, that eServices is perceived as "good enough to get by" by Senior Leadership.  Not intending to be cynical, but the impact of eServices' poor implementation on NHQ staff is a lot lighter than on the membership.  The pain isn't shared all that much. 

This is certainly my perception, and shared by others.

I am pretty certain eServices (and WMIRS) hurts retention and mission readiness - I don't think anyone walks away just because these apps are awful, but they do figure into the sum of frustrations.  I think Senior Leadership can truthfully say both apps work - because they do; they just miss the bigger picture that they are a major frustration point for members. There are a LOT of volunteer member hours being sucked up by these apps. Those hours come out of availability for other tasks. TANSTAAFL, right?

Please put that on a T-Shirt, and require this be discussed at every committee meeting!

Volunteer hours maybe "free", but they aren't unlimited by a long shot.  More important, member initiative and good feeling is even more limited.  Whether it's eServices, a uniform argument,
or some other more trivial nonsense that could be ended with a sentence but has been left open for a decade, every minute wasted wrestling a system or decision that should be a baseline
"given" is a minute (or hours) not spent building real plans, recruiting, or training, or anything else which is actual mission-focused.

I know that after I've spent time either wrangling the system or arguing with frustrated members about the way things "should" work, vs. the way they actually do,
I'm not particularly interested in digging down into an ops plan or other stuff that's actually my job - so that stuff falls to the side, and we never make any progress.

Rinse and repeat that times 1400 some charters and staff at each level.

[JeffDG]

As you may have guessed, I have definite thoughts on this matter, but it's likely that I don't meet the criteria of "someone who hasn't...", nor am I retired.

Participating in such an effort:  Count me in!  Love to pitch in and help.
PM'ing such an effort:  Even though it's what I do (have the PMP cert and everything!), I can't see giving it enough time/effort/attention.

[JeffDG]

I think, reading between the lines, that eServices is perceived as "good enough to get by" by Senior Leadership.  Not intending to be cynical, but the impact of eServices' poor implementation on NHQ staff is a lot lighter than on the membership.  The pain isn't shared all that much. 

This is certainly my perception, and shared by others.

I am pretty certain eServices (and WMIRS) hurts retention and mission readiness - I don't think anyone walks away just because these apps are awful, but they do figure into the sum of frustrations.  I think Senior Leadership can truthfully say both apps work - because they do; they just miss the bigger picture that they are a major frustration point for members. There are a LOT of volunteer member hours being sucked up by these apps. Those hours come out of availability for other tasks. TANSTAAFL, right?

Please put that on a T-Shirt, and require this be discussed at every committee meeting!

Volunteer hours maybe "free", but they aren't unlimited by a long shot.  More important, member initiative and good feeling is even more limited.  Whether it's eServices, a uniform argument,
or some other more trivial nonsense that could be ended with a sentence but has been left open for a decade, every minute wasted wrestling a system or decision that should be a baseline
"given" is a minute (or hours) not spent building real plans, recruiting, or training, or anything else which is actual mission-focused.

I know that after I've spent time either wrangling the system or arguing with frustrated members about the way things "should" work, vs. the way they actually do,
I'm not particularly interested in digging down into an ops plan or other stuff that's actually my job - so that stuff falls to the side, and we never make any progress.

Rinse and repeat that times 1400 some charters and staff at each level.

Wait, are you proposing an ICL to authorize this Committee to have a custom uniform with T-shirts?   

[NIN]

And in typical fashion, *bam* its a uniform discussion.

[JeffDG]

And in typical fashion, *bam* its a uniform discussion.

[Tim Medeiros]

Look, here's something that EVERYONE in CAP can do to help out the online system.
Test it.

When you find something "wrong", write up a detailed report for the IT people to look at.
They really can't use "hey this thing sucks"  or  "this this is broken"
However, they CAN use:
1.  a detailed description of what you were trying to do (i'm trying to add promotion data for a prior achievement to capid 123456)
2.  a detailed description of the steps you took leading up to the error
3.  the specific error, copied and pasted, or screenshot from your screen (so they can replicate it)
4.  it never hurts to throw in a suggestion for a fix, if you have that up your sleeve
5.  your operating system, browser+version

This is key for any support request, it makes ITs job a heck of a lot easier.

<snip>To that end, I think there should be a national IT 'committee' made up of volunteers.  And CAP's "CIO" should be a member at large.  This group would be responsible for clarifying a member-based, unit-based vision, mission, purpose, and methodology for CAP's IT structure.  The paid staff (full time and contractors) would then execute that on a M-F 8-5 basis.

There is an IT committee, however in my honest opinion, they are too far removed from the field at this point in time.

I am nearly a master rated IT in CAP (and it's my civilian job, at which i'm reasonably successful), and I can't tell you off the top of my head who CAP's chief information officer is, or name one thing I have seen that person (or their office) publicly do, ever.  If you want to know why the situation is what it is in CAP, start there.  And whoever that is, might just have their hands tied from upstairs, for all I know...  It's not an enviable position in the current climate.

The CAP ITO is Lt Col Bill Hughes of NYWG, he's been in the position as long as I can remember it being there.  Feel free to guess my opinion on that (note: he heads up the IT committee I commented on earlier).  NHQ/IT is headed up by Mr Joe Hall, I had the opportunity to meet him at a National Board meeting before he got the position (he was a dev at the time) he's a good guy.  The rest of the IT team from whom I've chatted with here and there, mostly online or at conferences, are good hard working people, they just have too much to do and too little time.  Not only are they working on member facing items, but also employee only, just remember that.

Also, coudano, great post, seems few actually try to 1) understand what is going on beyond their playpen, 2) realize just what it might take to go with the whole "scrap and rebuild better" plan.  I'd just add one thing, with eServices being on a .gov domain, there are an inordinate amount of rules and regulations that must be abided by on that front, which would add complexity, time and money, among other resources.

[JeffDG]

I'd just add one thing, with eServices being on a .gov domain, there are an inordinate amount of rules and regulations that must be abided by on that front, which would add complexity, time and money, among other resources.

Yet at the last CSAG meeting there was a proposal, referred to committee, to ban anyone other than NHQ from using anything but .GOV for their internet operations, showing yet another "disconnect" between the national leadership and the boots-on-the-ground.

So, if this goes forward, you'll have 6 months to shut down SERCAP.US.  An excellent use of volunteer time that would be.

[Eclipse]

So, if this goes forward, you'll have 6 months to shut down SERCAP.US.  An excellent use of volunteer time that would be.

Actually, this needs to happen and sooner then later, having 52 wings with separate infrastructure and domains, etc., is unprofessional, and borderline FWA for those
that are paying for it, not to mention that nothing says "professional" like an an email from "pilotcutie76@juno.com" with a pharma ad in the sig line.

Every member should get a wing.cap.gov email address when they join, and be required to use it for access to all CAP systems.  NHQ is already using GApps for cap.gov, so
it's just a matter of the number of licenses.

Create wing-level user admins, publish a set of templates for unit and activity use, and prohibit the use of other services for email and websites (which would also go a long way in the
fight against spam and shore up OPSEC).  Manage the exceptions as necessary, and there won't be many.

And then move on.

[JeffDG]

So, if this goes forward, you'll have 6 months to shut down SERCAP.US.  An excellent use of volunteer time that would be.

Actually, this needs to happen and sooner then later, having 52 wings with separate infrastructure and domains, etc., is unprofessional, and borderline FWA for those
that are paying for it, not to mention that nothing says "professional" like an an email from "pilotcutie76@juno.com" with a pharma ad in the sig line.

Every member should get a wing.cap.gov email address when they join, and be required to use it for access to all CAP systems.  NHQ is already using GApps for cap.gov, so
it's just a matter of the number of licenses.

Create wing-level user admins, publish a set of templates for unit and activity use, and prohibit the use of other services for email and websites (which would also go a long way in the
fight against spam and shore up OPSEC).  Manage the exceptions as necessary, and there won't be many.

And then move on.

Actually, Google will now permit unlimited free licenses to non-profits (up from their old limit of 3,000).  Now the issue is, they might treat a ".GOV" as a Google Apps for Government account, and then say "$50/user/year, no volume discounts", while a .org or other domain name can skate under 501(c)(3) rules.

Hell, if NHQ needs a provisioning script, I have one for TNCAP.US already that automatically provisions accounts and "Drive" rights based upon duty positions, give me a few days and I'm approximately 99% sure I could scale it (there are no limitations in the code at all).

And that's precisely what we have for TNWG...unit-level templates and events.  We are building a ton of capability. 

Heck, if NHQ would just give us a SAML gateway to eServices, I could get rid of users having to remember multiple passwords.

[Eclipse]

Actually, Google will now permit unlimited free licenses to non-profits (up from their old limit of 3,000).  Now the issue is, they might treat a ".GOV" as a Google Apps for Government account, and then say "$50/user/year, no volume discounts", while a .org or other domain name can skate under 501(c)(3) rules.

National is already using Google Apps for the cap.gov domain.  I don't know what flavor it is, hopefully it's the education flavor.  The .gov restriction indicated for the free accounts
is actually a non-issue once you substantiate your .gov email is tied to a legit 501(c)3, which is unusual but obviously not unheard of.   The indicated subdomain restrictions are also, obviously, a non issue.

My wing just (finally) converted our wing.cap.gov account to Apps for Ed this year.  No cost.

I first noticed NHQ  couple years ago when I added someone to my contact list with a custom picture and it asked me if I wanted to share the pic with the contact - that only works
on Gmail-Gmail hosted systems.   After that I noticed a number of NHQ websites, and a few sign-up forms were "Sites" or "Apps" hosted services.

So the main infrastructure is already there.

[JeffDG]

Actually, Google will now permit unlimited free licenses to non-profits (up from their old limit of 3,000).  Now the issue is, they might treat a ".GOV" as a Google Apps for Government account, and then say "$50/user/year, no volume discounts", while a .org or other domain name can skate under 501(c)(3) rules.

National is already using Google Apps for the cap.gov domain.  I don't know what flavor it is, hopefully it's the education flavor.  The .gov restriction indicated for the free accounts
is actually a non-issue once you substantiate your .gov email is tied to a legit 501(c)3, which is unusual but obviously not unheard of.   The indicated subdomain restrictions are also, obviously, a non issue.

My wing just (finally) converted our wing.cap.gov account to Apps for Ed this year.  No cost.

I first noticed NHQ  couple years ago when I added someone to my contact list with a custom picture and it asked me if I wanted to share the pic with the contact - that only works
on Gmail-Gmail hosted systems.   After that I noticed a number of NHQ websites, and a few sign-up forms were "Sites" or "Apps" hosted services.

So the main infrastructure is already there.

Well then, hell yes, that's a great solution!

I would caution against the "only cap.gov" addresses thing...even with the stuff we're doing we only get about 10% of folks sign in in a given month, and ~15% have signed on to their @tncap.us account, ever.

We do, however, have mailing lists that include member's primary e-mail address (drawn from eServices) that work fantastically well...also auto-provisioned.

For OPSEC reasons, we only permit "Drive" rights to internal accounts, but e-mail going out is less bad. 

A better approach would be a "carrot" vs. a "stick" however.

[Eclipse]

A better approach would be a "carrot" vs. a "stick" however.

I would tend to agree, however the sad reality is that in CAP a lot of people won't do anything unless you force the issue.

One of my old units walked this path - requiring people to only use their CAP email for CAP business.  It was an uphill climb,
but after the initial gnashing of teeth, people got over it.  Not 100%, but the ones you really care about.

These days there are plenty of easy ways to consolidate email (hint: if you have to use more then one email client for >all< of your messages,
you're doing it "hard", if not wrong.  The occasional stick-in-the-mud IT shop notwithstanding).  I Have something like
8 email addresses from different companies and organizations all consolidated seamlessly in my primary gmail domain.

And no spam, either - now there's a useful wing conference breakout - "Consolidating your email accounts".

Anyway - the amount of time that could be freed up, not to mention money and other resources with this large but relatively simple project
is pretty significant.

[NIN]

Actually, Google will now permit unlimited free licenses to non-profits (up from their old limit of 3,000). 

"I was not aware of that!"

Thats encouraging. Now build an LDAP provisioning setup from eServices..

[JeffDG]

Actually, Google will now permit unlimited free licenses to non-profits (up from their old limit of 3,000). 

"I was not aware of that!"

Thats encouraging. Now build an LDAP provisioning setup from eServices..

Like I said, I've cracked that nut already...at least as far as provisioning.

Not LDAP, I'd actually prefer SAML

[Tim Medeiros]

I'd just add one thing, with eServices being on a .gov domain, there are an inordinate amount of rules and regulations that must be abided by on that front, which would add complexity, time and money, among other resources.

Yet at the last CSAG meeting there was a proposal, referred to committee, to ban anyone other than NHQ from using anything but .GOV for their internet operations, showing yet another "disconnect" between the national leadership and the boots-on-the-ground.

So, if this goes forward, you'll have 6 months to shut down SERCAP.US.  An excellent use of volunteer time that would be.

Just to point out, that proposal was actually authored by Col Webb, the cap.gov domain administrator.  He is another that I have an issue with.  When trying to setup a system to work with Google, his response to a clearly DNS issue was to use a "standard email provider".