Welcome, Guest. Please login or register.
Did you miss your activation email?
November 20, 2018, 01:41:45 AM
Home Help Login Register
News:

CAP Talk  |  Cadet Programs  |  Cadet Programs Management & Activities  |  Topic: Why aren't the Milestones proctored on a computer?
0 Members and 1 Guest are viewing this topic.
Pages: 1 [2] 3  All Print
Author Topic: Why aren't the Milestones proctored on a computer?  (Read 5278 times)
lordmonar
Too Much Free Time Award

Posts: 10,655

« Reply #20 on: December 13, 2012, 06:04:13 PM »

I think the reason is because they want to keep the milestones closed book, and if they were online they could say closed book but not enforce it.

This would happen at the unit as it does currently. Just on a PC.
Back in the day...I got a web based testing program...loaded the first 10 or so test into it.  And i proctored the cadets as the took the tests "on line".  When they were done....it instantly graded it and told them what questions they missed and what chapters to study....just like we are supposed to do on paper.

The reason why we don't do this...probably no one has ever proposed it to the CP guys as doable....and IIRC there was a lot of push back from the field about going to online testing.
Logged
PATRICK M. HARRIS, SMSgt, CAP
Pacific Region
krnlpanick
Seasoned Member

Posts: 256
Unit: RMR-CO-143

Yet Another Developer's Blog
« Reply #21 on: December 13, 2012, 06:24:04 PM »

Aux already covered how it could be an online secured test.

While I am not necessarily a glory hog, I would like to state for the record that Aux was agreeing with my idea I believe :)
Logged
2nd Lt. Christopher A. Schmidt, CAP
Pylon
Administrator

Posts: 5,132
Unit: NER-NH-038

Michael Kieloch, Marketing Communications & PR Leadership
« Reply #22 on: December 13, 2012, 06:29:39 PM »

The Marine Corps uses Questionmark Perception software for online proctored exams.  It requires the student log-in to the regular web course and select a test, at which point it prompts for a valid proctor's login, and then launches the Perception software.  The software locks the user from closing out or minimizing or otherwise exiting the test window (among other things) preventing the user from flipping between windows or programs or having the textbook or Google open in the background.  It obviously doesn't stop users from using notes, their phone, or another computer but that's why the proctor has to enter their login -- theoretically the proctor is present, but knowing the workstation is at least locked down, they don't have to be watching every single screen at every moment.  They really only have to ensure students don't have phones or notes or other devices out.  Makes it easier to proctor a roomful at once, if need be.
Logged
Michael F. Kieloch, Maj, CAP
Concord Composite Squadron, NH       
coudano
Salty & Seasoned Contributor

Posts: 1,125

« Reply #23 on: December 13, 2012, 06:44:21 PM »

How about a hybrid solution?
PDF's of the individual test booklets opened on a squadron computer and an all purpose answer sheet onto which the cadet can write their answers?
No test storage required, no internet required, answer sheet still in hand for hand grading.
Just make sure the PDF's are secured on the testing officers or squadrons laptop.

You are now making security of the laptop an issue.
It's quite frankly easier (and more secure) to lock paper tests in a file cabinet, than to try and guarantee a computer's security.
Logged
krnlpanick
Seasoned Member

Posts: 256
Unit: RMR-CO-143

Yet Another Developer's Blog
« Reply #24 on: December 13, 2012, 07:06:11 PM »

You are now making security of the laptop an issue.
It's quite frankly easier (and more secure) to lock paper tests in a file cabinet, than to try and guarantee a computer's security.

FWIW - there is no reason you couldn't lock that very same laptop up in the very same filing cabinet; as a matter of fact that is exactly where our squadron's laptop lives.

That being said, you have far better control of an electronic test than you do of a traditional paper and pencil test:

1) Question randomization - How many school exams have been passed by kids memorizing the order of answers from a leaked or stolen answer key?
2) Larger question bank - Further control can be gained by offering different wordings and formats of the same questions so that simple memorization can not be used to pass the test - requiring students/cadets to actually understand the material they are testing on
3) Maintainability - Curriculum that is static is stagnant. Knowledge is an organic lifeform of it's own that is ever evolving - having the test and testing materials available online allows easier maintenance of that material
4) Cost - Whether it be cost to distribute printed materials or perceived cost of time spent by the testing officer grading tests, it is simply more cost effective to proctor exams electronically.
5) Accuracy - Even testing officers make mistakes every once in a while, but a computer does exactly what it is programmed to do.

As for the security element of electronic testing, electronically proctured tests are no more or less secure than paper tests - the security issue with most tests is the human element in most cases. If the testing officer leaves the paper answer key sitting on his desk, that is just as much a security issue as that same testing officer leaving the laptop sitting on the desk unattended. Also, with the added control of the electronic test, even if someone sees the test before taking it, it is highly likely that if it is done correctly - it will be a completely different test when they take it.
Logged
2nd Lt. Christopher A. Schmidt, CAP
EMT-83
Salty & Seasoned Contributor

Posts: 1,887

« Reply #25 on: December 13, 2012, 07:35:55 PM »

You are now making security of the laptop an issue.
It's quite frankly easier (and more secure) to lock paper tests in a file cabinet, than to try and guarantee a computer's security.

FWIW - there is no reason you couldn't lock that very same laptop up in the very same filing cabinet; as a matter of fact that is exactly where our squadron's laptop lives.

As long as itís in a separate drawer in that filing cabinet, and not secured with the tests.
Logged
krnlpanick
Seasoned Member

Posts: 256
Unit: RMR-CO-143

Yet Another Developer's Blog
« Reply #26 on: December 13, 2012, 08:25:22 PM »

As long as itís in a separate drawer in that filing cabinet, and not secured with the tests.

Actually, to be entirely accurate ours is in it's own filing drawer in a completely different filing cabinet than the paper tests; but I digress.
Logged
2nd Lt. Christopher A. Schmidt, CAP
cm42
Recruit

Posts: 38

« Reply #27 on: December 13, 2012, 10:06:44 PM »

You are now making security of the laptop an issue.

It's quite frankly easier (and more secure) to lock paper tests in a file cabinet, than to try and guarantee a computer's security.

FWIW - there is no reason you couldn't lock that very same laptop up in the very same filing cabinet; as a matter of fact that is exactly where our squadron's laptop lives.

Much easier to hook the USB drive up, copy the PDFs, and be out before anyone notices. Brute force crunching on the PDFs and they'll be out in the open in no time.
Logged
Майор Хаткевич
200,000th Post Author
Salty & Seasoned Contributor

Posts: 6,059
Unit: GLR-IL-049

« Reply #28 on: December 13, 2012, 10:20:03 PM »

As long as itís in a separate drawer in that filing cabinet, and not secured with the tests.

Actually, to be entirely accurate ours is in it's own filing drawer in a completely different filing cabinet than the paper tests; but I digress.

Yep. It's an interesting rule, but it works.
Quote
1-5. Testing Material Storage. All hard-copy testing materials will be stored in a lockable (key or combination) metal cabinet. It can be a safe, filing cabinet or standing cabinet. Nothing other than testing materials can be stored in the locked area.
Logged
lordmonar
Too Much Free Time Award

Posts: 10,655

« Reply #29 on: December 13, 2012, 10:50:35 PM »

You are now making security of the laptop an issue.

It's quite frankly easier (and more secure) to lock paper tests in a file cabinet, than to try and guarantee a computer's security.

FWIW - there is no reason you couldn't lock that very same laptop up in the very same filing cabinet; as a matter of fact that is exactly where our squadron's laptop lives.

Much easier to hook the USB drive up, copy the PDFs, and be out before anyone notices. Brute force crunching on the PDFs and they'll be out in the open in no time.
assuming they are willing to do that....having paper tests locked up will not deter them from cheating.
Logged
PATRICK M. HARRIS, SMSgt, CAP
Pacific Region
Extremepredjudice
Banned

Posts: 1,452
Unit: Freebooter

« Reply #30 on: December 13, 2012, 10:54:17 PM »

You are now making security of the laptop an issue.

It's quite frankly easier (and more secure) to lock paper tests in a file cabinet, than to try and guarantee a computer's security.

FWIW - there is no reason you couldn't lock that very same laptop up in the very same filing cabinet; as a matter of fact that is exactly where our squadron's laptop lives.

Much easier to hook the USB drive up, copy the PDFs, and be out before anyone notices. Brute force crunching on the PDFs and they'll be out in the open in no time.
Wouldn't be hard to get the paper tests, either.
Logged
I love the moderators here. <3

Hanlon's Razor
Occam's Razor
"Flight make chant; I good leader"
SarDragon
Global Moderator

Posts: 10,470
Unit: NAVAIRPAC

« Reply #31 on: December 14, 2012, 04:56:57 AM »

One of the problems I've seen locally is that units can have a meeting place, but with no facilities such as 'Net access, storage, etc. Bring everything in before the meeting; take it out at the end. That makes it really hard to do all these kool things everyone is talking about. In this case, nothing beats a paper test.
Logged
Dave Bowles
Maj, CAP
AT1, USN Retired
Mitchell Award (unnumbered)
C/WO, CAP, Ret
Cool Mace
Seasoned Member

Posts: 351

« Reply #32 on: December 14, 2012, 10:43:16 AM »

Aux already covered how it could be an online secured test.

While I am not necessarily a glory hog, I would like to state for the record that Aux was agreeing with my idea I believe :)

My apologies, good sir.

:AHEM!: I believe Krnlpanick already covered  how it could be an online secured test.
Logged
CAP is what you make of it. If you don't put anything in to it, you won't get anything out of it.
Eaker #2250
C/Lt Col, Ret.
The cookies and donuts were a lie.
Майор Хаткевич
200,000th Post Author
Salty & Seasoned Contributor

Posts: 6,059
Unit: GLR-IL-049

« Reply #33 on: December 14, 2012, 01:50:47 PM »

Yes,

eServices is the EASIEST route to go. The issue becomes one if infrastructure.

Now, what they COULD do, is provide the test online to the Testing Officers as an option within every cadet's testing record. The TO would PRINT a generated test from a question bank, and once the cadet is done, the TO would enter it into the system. The system would grade the test, give out a score, and a printable reference page for missed questions. If the cadet did not pass, then the next time the test is printed, the questions are different. Since the Award tests are based on the Achievement tests, they could then use the cadet's past testing history, and even make the test MORE meaningful by throwing in the questions cadets may have missed in the past. This would ensure that they aren't "just passing" but studying up on what they miss as well. There's a lot that can  be done on the back end of a system like this to make the program stronger.

It may sound like a lot of work, but it's honestly no different than what I do now (Look at the key and mark their answer sheet), manually get their score, and manually write out what they need to work on.

This would also make the tests 100% more secure, as not even the TO would have the keys, only National. Still a paper test, but graded by NHQ.
Logged
a2capt
300,000th Post Author
Salty & Seasoned Contributor

Posts: 5,089
Unit: pǝʇɹǝʌuı

« Reply #34 on: December 14, 2012, 01:53:51 PM »

They have spent the better part of the last 10 years trying to -get away- from grading tests.

You want to wait for WB results now? Then that makes the "Correct to 100%" part a multi week process, too, as when most units have meetings, NHQ is closed.
Logged
Майор Хаткевич
200,000th Post Author
Salty & Seasoned Contributor

Posts: 6,059
Unit: GLR-IL-049

« Reply #35 on: December 14, 2012, 01:54:52 PM »

They have spent the better part of the last 10 years trying to -get away- from grading tests.

You want to wait for WB results now? Then that makes the "Correct to 100%" part a multi week process, too, as when most units have meetings, NHQ is closed.

I wasn't aware that the Achievement tests are graded manually by someone at NHQ...
Logged
krnlpanick
Seasoned Member

Posts: 256
Unit: RMR-CO-143

Yet Another Developer's Blog
« Reply #36 on: December 14, 2012, 02:22:01 PM »

They have spent the better part of the last 10 years trying to -get away- from grading tests.

You want to wait for WB results now? Then that makes the "Correct to 100%" part a multi week process, too, as when most units have meetings, NHQ is closed.

I wasn't aware that the Achievement tests are graded manually by someone at NHQ...

I believe that Aux was insinuating that the *system* would grade the tests, and the results would be locked up at NHQ (in the database) rather than in each TO's filing cabinet. I see no manual process here for NHQ at all other than implementing the process. I think this is a pretty good compromise wrt my suggestion as well. Either way would have the same benefits and make the program stronger. Really the big difference is keeping them as "paper tests" for the cadets or moving them online - the end result of either is exactly the same. The current weakness is that the tests are static, difficult to maintain, and subject to human err in grading. While there is still the possibility of human err in "fat fingering" the input by the TO into the system using Aux' strategy it is IMHO less than the possibility of human err in manually grading the test.

Just MHO.
Logged
2nd Lt. Christopher A. Schmidt, CAP
a2capt
300,000th Post Author
Salty & Seasoned Contributor

Posts: 5,089
Unit: pǝʇɹǝʌuı

« Reply #37 on: December 14, 2012, 02:25:37 PM »

I wasn't aware that the Achievement tests are graded manually by someone at NHQ...
No.. but..
This would also make the tests 100% more secure, as not even the TO would have the keys, only National. Still a paper test, but graded by NHQ.
You elluded to that kind of system here. So I responded, and "someone" to me, means a person, not a system/computer. Which means open for business the next day kinda delay.
Logged
Майор Хаткевич
200,000th Post Author
Salty & Seasoned Contributor

Posts: 6,059
Unit: GLR-IL-049

« Reply #38 on: December 14, 2012, 03:10:02 PM »

While there is still the possibility of human err in "fat fingering" the input by the TO into the system using Aux' strategy it is IMHO less than the possibility of human err in manually grading the test.

Easy fix. After submitting, the system could ask the TO to check that they marked the correct (the cadet's actual) answers for whichever are wrong. That's a double check that should resolve 99% of that problem.

I wasn't aware that the Achievement tests are graded manually by someone at NHQ...
No.. but..
This would also make the tests 100% more secure, as not even the TO would have the keys, only National. Still a paper test, but graded by NHQ.
You elluded to that kind of system here. So I responded, and "someone" to me, means a person, not a system/computer. Which means open for business the next day kinda delay.


I assumed that when I'm talking of eService's tests, "graded by NHQ" implies the same way they grade current online tests. It's 2012. I don't expect anyone having an online system with a manual grader.
Logged
Cool Mace
Seasoned Member

Posts: 351

« Reply #39 on: December 14, 2012, 03:19:37 PM »

Quote
Easy fix. After submitting, the system could ask the TO to check that they marked the correct (the cadet's actual) answers for whichever are wrong. That's a double check that should resolve 99% of that problem.

I like this idea. That would help a lot when it comes to making sure the "wrong" answers are really incorrect, and not just human error on the TOs part.
Logged
CAP is what you make of it. If you don't put anything in to it, you won't get anything out of it.
Eaker #2250
C/Lt Col, Ret.
The cookies and donuts were a lie.
Pages: 1 [2] 3  All Print 
CAP Talk  |  Cadet Programs  |  Cadet Programs Management & Activities  |  Topic: Why aren't the Milestones proctored on a computer?
 


Powered by MySQL Powered by PHP SMF 2.0.14 | SMF © 2017, Simple Machines Valid XHTML 1.0! Valid CSS!
Page created in 0.372 seconds with 25 queries.
click here to email me