Team 2 Week 8 Blog

 

This week our beneficiary discovery focused on A/B testing this week’s MVP and talking with internal problem solvers. We also continued to do some dual-use and outside user research.

 

Key takeaways:

1. Removing data entry redundancy is high priority for making our product viable. The program essentially must be a one stop shop.
2. Current systems really are a “pandora’s box” of spreadsheets. System must be able to centralize all of them.
3. Improved user interface will make teams more likely to input information. Has to be simple and only a few clicks.
4. Would be great if program could output MICON documents such as the cost estimate worksheet

 

1. Cpt. Nick – Kessel Run

• Often times the higher ups care about increased combat capability
• Say that with this saved time manually entering things they can spend that time thinking instead
• Maybe create new metric that allows them to measure their progress better(ex. What percent of METL tasks are at a T rating after these courses)

2. Major Ryan – Battalion XO

• If we created a course catalog, with the ten(or more) most frequently used courses with the info we’ve been putting on the MVPs(cost, location, forms, etc.), made a tool that would generate the CEW from historical data(seems like an important feature to add), and added the ability for them to add more courses, that would help the ODAs a lot.
• It also sounds like the best course for us is to make a simple, easy-to-use form that one of Ryan’s captains can then use to fill in the historical data since we cannot
• Once the app is approved onto NIPR then the app must be added to their portal so that the right people can use it. To find more information about IT specifications for the portal we need to talk to Christian Cordero, the battalion portal manager(already on the engagement tracker) and for more info talk to the guys who work in RJ’s shop.

 

3. Captain John (Ret. USMC)

• Every platoon commander would dream of having a tool like this, because they can click submit and go train. But inevitably, there is some sort of sticking point—division has to get all flu shots, or someone’s mom is sick, or battalion has to meet certain standards of dental health. So much stuff happens in the margin that thinking you can reliably predict farther out than a couple of days is foolish.
• Your MVP will only work if multiple people the slap the table and say “we need this.” The key to success, in my mind, would be if you had a team from colonel to captain that embraced it, and if the next group of dudes that come in adopted it.
• A lot of guys wind up recreating the wheel once they’re in command, and each guy uses the spreadsheet he used when he was a platoon leader. And it’s always under the comment of “Oh man, this is going to be so much easier to use.” I find it very difficult to predict an end to it. On one hand you want innovation and iterative improvements if the system’s not working, but it becomes a pandora’s box of spreadsheets.

 

4. Nelson (Lawyer)

• This could apply in law, but it’s not necessarily for the lawyers themselves. We have an internal team at the firm making sure we’re up to date on continuing ed requirements. Secretaries especially are in charge of this.
• That said, sometimes lawyers get down to the wire and say “let me knock all of these hours quickly,” meaning they go to a conference at the last second.
• People do just google potential conferences, rather than thinking about what specific courses would lead them to improve.

 

5. Captain David (Former ODA Commander and AS3)

• In an ODA, the Captain is only there for 2 years—Warrant Officers are where the continuity is, 6 years. They should be the ones to go through the BFA, ideally. If Warrant is absent in some way, it hurts. The team sergeant also has the knowledge of what’s out there in terms of training, since he’s been in SF for 8 yrs by the time he’s an E8. Information about training gets passed down ancestrealy, not codified.
• HSF group has a “Lessons Learned” guy. After every time we do a major training event off-post, every team leader does a AAR, and “LL guy” is supposed to aggregate these. I didn’t utilize this as a team leader, but once you get to battalion, you realize how important it was.
• Be sure to look at DTT—it’s a mandatory practice, and the official channel for me to submit things up to higher. As a leader, I let me warrant handle it. I didn’t get to involved. When I was in battalion, it became my world. I’d challenge younger team leaders to use DTT now. When I went to staff, I was much much more involved in it, it let me tell what each team is doing, and know what battalion can do for them. Every day I’d go through postings of DTT, and see what was new.

 

6. Sgt Guzman

• Regular problem they run into is ODAs trying to order supplies at the last minute before a course.  It would be great if the app had mechanisms to address this.
• Equipment lists for courses could be sourced from a few places:

• • The vendors often state what equipment is supplied vs what the ODAs need to bring.
  • S3 could run reports of past purchases

• He keeps a log of basic shopping items – such as chem lights – the ODAs request and uses that to forecast purchases over the next six months.

 

7. Chief Officer

• Having the Risk Assessment for a given training is a good idea as it could be reused by each ODA.

• Consider adding the Nearest Medical Facility, its level of care as Accident Plan needs to be part of each training.

• Consider having the Accident Plan as part of the course. Needs to allow for a place for the ODA point of contact while training.

• The METL associated with the training should be listed.
• SOCForwardNWA (Classified system) is an online risk assessment generator tool that publishes the risk assessment to an approved PowerPoint template.  That would be a great model to replicate for training.

 

8. Kati Samerigo

• AWS is a familiar platform for the Army.  Look for the FEDRAMP certification for any cloud technologies/platforms you might want to use.  Level 5 is for unclassified. Level 6 is for classified.
• SOCOM out of Tampa would be the petitioning entity we would need to sponsor internal accreditation of our application.
• Interoperability of systems is not a significant army focus so whether APIs exist for touchstone systems such as DTMS will primarily be a function on whether the civilian version – if it exists – of that software had APIs.

 

9. Chief Matlock (ODA NCO)

• Final evaluation before deployment focuses primarily on Shoot, Move, Medicate, and Communicate. i.e. It does not necessarily tie to all of the METL the team was focused on over the training cycle.
• DTMS is administered and referenced at the battalion level. ODAs will continue to need their own view of the team that is currently accomplished via spreadsheet.
• The sample BFA is for a team that has to completely rebuild and it encompasses all the different ODA specialty types.  No single BFA would be that extensive. (Scheduled follow-up meeting to walk it in detail.)

 

10. Captain Mike – Charlie Company BATXO

• What would be mindblowingly amazing- i click on training portal x, and click on new training. And it populates the training card. It has a box for micon required- if yes,  ability to input everything required for a MICON. If MICON no- then just the format required for training card. That would be fucking epic.
• Not much use to the simple style MVP
• Mike told us all of the information used in the training tracking system they are using in Charlie company

 

Team 2 Week 6 Blog

 

This week our beneficiary discovery focused on retesting new aspects of our MVP with contacts we already had interviewed and continuing to seek out civilian applications.

Interview count this week: 8

Interview count last week: 17

Key takeaways:

1. Civilians that we have interviewed are very excited about the potential of our MVP
2. MVP needs to replace or work with the cost estimate worksheet, as it is used for every training evolution and doing it twice would just be more work
3. We should look into other military training resources such as DTMS and RFMSS to see how our product can work within or supplant them.

 

1. Cpt. Chris – ODA commander

• The ODA Weapons Sgts are the best people to discuss planning weapons training, booking range time, ammo needs
• Each ODA has a slightly different team profile workbook. Will share our team’s template.
• Different teams have different levels of historical training and MICON repositories to fall back upon.

 

2. Chief Joe – ODA warrant officer

• Most ODAs do only one MICON per training cycle. CREF ODAs do 7-8.
• Their company has an archive of MICONs. Takes him about 4 hours to turn a MICON since he almost certainly has a recent example of a similar training plus this is his full time role.  
• DTMS is the Army’s training management system for individual training tasks. SF has avoided using the system to date but it was recently mandated within SF as well. It will likely take the place of the team profile spreadsheets – or at least parts of them.

 

3. Major Ryan – Battalion XO

• Hardest part is planning when to schedule training events based off long range calendar (especially when deployment times change) Second hardest is filling out the cost estimate worksheet(must be done for both on and off base training)
• Two documents that must be filled out regardless of on or off base:
  • CEW
  • Risk assessment(difficult to tackle could be useful to auto fill later) which describes what risks lie in training and how they will mitigate those risks and then list residual risks

 

4. Cpt. John

• RFMSS is a database of all the range coordination to see other units that are using that area. You cannot use a training area until it has been approved and deconflicted against other training units. All of the US military bases have joint service training. If Army has it marked down on Feb 23rd, I (as a Marine captain) can’t use it.
• A savvy infantry officer or ODA captain uses RFMSS as a predictive tool to forecast training out 3-4 months. This is very important because worst thing that happens is last-minute friction of a RFMSS conflict.
• RFMSS can be a very powerful tool, but there a bunch of flaws. User experience isn’t great, but the idea is right.

 

5. Cpt. Josh – Battalion S4

• Going under budget should never be a concern, if the teams go under budget there are numerous things that the battalion would like to buy that they normally don’t have the money for.
• Would like a total cost and a way to input final costs of training
• System must include all items on a cost estimate worksheet so that work doesn’t need to be done twice

 

6. RN Anna

• Nurses are required to keep up with 20 continuing ed requirements to stay certified, but most people wait until the month before there certification is up and do it all online.
• I’m sure if I spent a bunch of time, I could find 20 hours of free courses, but because of my time crunch, I’d rather just pay $20 to have them all in one place.
• As it currently exists, the process is very easy, but not particularly high quality. I love this (MVP). It probably would save a lot of time and is convenient. If the hospital would provide me with a list what people have done before, that would save me even more time and I could see what is a credible class.

 

7. Firefighter EMT Bobby

• Every firefighter is at least an EMT if not a paramedic.The way we do it (through the national registry), it sucks. If you don’t upload it there, if you try to upload your CE’s at once, it gives me heartache trying to stay up to date.
• The current process is probably the hardest thing. I’m going to let my national registry lapse because I have to jump through hoops, and some classes don’t count. I worked hard for my certification, but it’s too much of a pain in the butt to upload and if it’s not up to the national registry’s standards—if they don’t like it, or who taught it– they don’t count it. County and state are lot more lenient about what they’ll accept.
• This MVP would be beautiful for us—I can see applying it to airways, breathing, circulation. It’s easily read, “I need this class, so I check this box, and add it to my shopping cart.” With the National Registry, you’re left guessing. This would clarify a lot of things, because I now could know if didn’t get enough X recert hours and make a decision accordingly.

 

8. Lieutenant Matthew (Fire)

• In California, opting to train through the CalJAC system pays money back to your department. Some departments throughout the state love it, and some don’t sign up for it.
• With CalJAC, the individual firefighter only gets additional money if they use the GI Bill.
• The budget side of training is really not a big concern for us. We do most of our training in-house, and if we want to go outside, someone reciprocates with us. In the Fire community, we’re willing to train others if they’re willing to learn. We’ll make it happen.

Team 2 Week 5 Blog

 

This was a very successful week of beneficiary discovery. We stacked up 17 interviews and were able to talk with many ODA commanders, the beneficiary we have identified as most important. We further developed our MVP and presented it to members of 3rd group 2nd battalion on a site visit to Fort Bragg, where we were able to receive direct, in-person feedback as well as develop relationships with contacts for the future. We also spoke to members of the education and emergency response industry to feel out the possible applications of our future product towards other industries.

 

Summary of Key Insights and Trends:

• Tested and debunked the concern that ODAs may view their needs and training events as unique snowflakes that would lead them to reject a recommendation engine or training catalog.
• Tested the concept of Amazon ratings. They were highly enthusiastic about the potential value it brings.  They did not feel it imposed a meaningful amount of additional work for them to enter ratings.
• Tested the level of pain ODA commanders feel with the current process. We were concerned that ODA commanders had not been expressing a sufficient amount of pain around the current process during our calls. When we were face to face, they were highly engaged, offering lots of suggestions and anecdotes, and asking lots of questions. Consistently, people rated the current process 4 or 5 with 5 being the most common response.
• Tested whether there was value in the proposed system.  Everyone believed that if the system performed as described it would dramatically reduce the level of pain and provide additional benefits.

 

1. Sergeant First Class RJ – Communications Specialist

• Hardest parts in filling out the worksheet are estimating costs and gaining approval from superiors. Sometimes they inquire whether training could be completed cheaper.
• Good reaction to MVP. Mentioned that everyone gets stressed around budget deadlines as the current process is a “nightmare”
• “Things that pre filled out are always good.”
• If a webapp, then we must first host the website ourselves and then if the military likes it they will acquire and eventually run the webapp internally

 

2. Mr. Sean, Charter School Network Talent Development Team Operations Manager

• The tool is very useful, and can solve a very clear need for tracking teacher professional development.
• Great improvement on current system. If the whole charter network could feed into populating the system, you’d gain more at scale.
• Allowing teachers or grade/department chairs to self-assess and request the training they want makes people feel like they own their PD more. That keeps them more engaged at work, and benefits the school. That said it it’s incumbent on the leader to follow-up when the teachers don’t get the training they requested.
• This easily populates a ranked budget– once you reach the limit of the funding you have for teacher training, you can cut it off.
• Who’s going to populate the catalog material? If other teachers go out and find great PD, then they can share it with their coworkers.

 

3. Captain David – ODA Commander

• The current tool is frustrating in that there are redundant pieces, plethora of requirements that are out there. Copy and Paste is the right answer, but no one puts in the dates. If the historical data can populate, and fill in with accurate dates, it’ll be an improvement.
• Risk assessment is always hot button– the training is always the same, but the risk assessment is an additional step that must be done each time.
• There’s always talk of creating this mythical document of translating dollars spent into readiness. Maybe having a team focused on it full-time can finally make it happen.
• This gives me a place to start with the budget, but beyond that I don’t see how this helps with completing MICONs.

 

4. Captain Jim – ODA Commander

• Big pain is finding approved vendors and keeping budget for a vendor below $25,000, the cutoff at which you must write up a contract with that vendor in order to use them
• Another big pain is not knowing what documents must be approved before certain training tasks. Told story about already having parachute on his back for jump training and receiving a call that he hadn’t filed some necessary paperwork.
• Providing an easy way to access AARs would be great for solving both of these issues

 

5. Captain Chris – ODA Commander

• Having a list of approved vendors would be huge
• No system exists to catalogue training shortfalls for individuals or teams
• Connecting money to readiness will be difficult because of how diverse special forces training is, but it would be very useful for keeping those approving the budget accountable for how ready their teams are

 

6. Dr. Tony – EMS

• EMS is required to have recertification every two years or so. This includes either in-service training or going to conferences.
• Volunteer and small-state agency EMS teams have to pay for their training out of pocket.
• “Folks have been looking for something to track continuing education for a long time. The difficulty I see is that it’s late in the game. With what you have here, for volunteers, anyone managing and paying for their own con ed, it helps. Proactively helps decide not only where I’m going to get my hours but what it’ll cost me as well.”

7. Cpt. Leonard – ODA Commander

• The current process of making the yearly plan and budget is extremely painful. He rates it 5 of 5. It also regularly requires nights and weekends.

• His ODA has a consistent mission assignment. As a result, there is definitely opportunity for that ODA to recycle significant portions of past training plans.

• He also believes there is significant overlap in the training activities of the various ODAs. Consequently, there is opportunity and value in a catalog of training.

• The proposed system – if it truly works as described – would dramatically improve the process.

8. Chief Villanueva – Group Maintenance Chief

• The ODAs definitely regularly reuse training events.
• For the teams he has worked with, the chief and commander solicit bottom-up feedback from the ODA for the next training cycle. They then pull-out the various METL guidance and decide on how to orient the team within the guidance incorporating the ODA feedback.  From there, they make the plan to bring back to the ODA.
• Current process is a 5 of 5 on the pain scale.
• The proposed system would be great to have.

9. Cpt. Diaz – Assistant S3 (Current Operations)

• Cpt. Diaz clarified for us that the Battle Focus Analysis does not come before the annual budget and training plan. They are integral parts of it.

• Liked the direction of upfront questions to shape the training needs but it is far too simplistic. Ideally, the system would display the various levels of METL guidance and a list of all personnel and their individual training records against that METL guidance. From this view, the team leadership could identify what is due to be hit again, what needs to be hit given the upcoming mission profile, and what the team will ignore b/c there is not time.

• The next step is to try laying that out on the calendar integrating mandatory and requested leave, assigned duties and trainings, and course availability. This forces a second round of compromises and priority juggling.

• His team keeps an Excel spreadsheet laying out the profile of each team member.

• Current system is a 5 for pain.

• His team not only does the annual budget and plan but immediately knocks out all of the MiCons up front. It is A LOT of work but just makes things easier going forward. [This is practice is not widely followed but has significant benefits. If we added the MiCon support, it would confer many of these benefits to all of the teams almost automatically.]

10. Maj. Morgan – Company Commander

• Could the system capture training results? (It could but would people use it or view it as a burden?) Depends upon the scope and intention but there are some definite applications for both the ODA and senior leadership worth exploring.

• This system could be powerful both in terms of time savings but also the sophistication it could offer. It could pose risks if over time its guidance is increasingly viewed as doctrine vs suggestions. [Related to the fact that the system would be recommending training activities. Not only the ODAs but the entire organization would need to internalize and maintain an appreciation that the recommendations are not gospel. Note to self: Build specific wording/warning into the system reinforcing this.]

• Who would have to enter this historical information? (Not the ODAs but we have to prove that it is indeed possible to compile the information.)

11. Chief Matlock – Assistant Detachment Commander

• He has a nonclassified BFA training deck and sample MiCon he will share with us.

• He charted out the components of the BFA and what they identify.

• Loves the training catalog concept as well as the Amazon ratings. He noted they have some vendors that have ongoing deficiencies and the ratings could help police that.

• He pulled me to the side in the hallway to provide more insights and convey that he sees enormous potential in the system overall.

12. Cpt. Barney – Signals ODA commander

• The Signals ODA has some specialized needs. [We need to follow-up with him for a longer discussion of those unique needs.]

• Current process is painful 5 of 5. Sees a lot of opportunity in the tool.

13. Sgt. Guzman – Supply Sergeant

• Works on the supply side and was extremely engaged and curious about whether the system could help support equipment needs.

• He notes that some equipment has to be replaced on a regular schedule but the request still usually comes as a one-off, frequently last minute, request.  E.g. Climbing ropes. They need to be replaced yearly, even if they have not been used, when a team goes to the climbing course.

• Numerous other items are similar. Recommended also following-up with Reposo and Fontanez on this topic.

14. Cpt. Fleming – Company Commander

• Loved the Amazon ratings. He cancelled a training last cycle because he just happened to hear that another ODA went to the same training course and said it was garbage.

• Current process if 5 of 5 on pain scale.

•  Echoed the comments on profiling the team against the METL.

• Definitely feels there is meaningful overlap amongst ODAs making both a catalog and recommendation engine feasible.

15. SM Croft – Company Sergeant Major

• Likes the Amazon ratings and the idea of course catalog. Commented they just sent two people to Germany for French training b/c the Army schools did not have availability at the right time.  Having a list of alternate providers would be great.
• There is absolutely overlap in training profiles which make both the catalog and recommendation engine feasible for real world deployment.

16. Maj. Schubert – Battalion XO

• Showed us the training card system used by one of his companies. It is not SOP but it makes it easy to track the results of training events. The current system is paper-based which means you have to consult the physical binders. It would be great if it was electronic.

• Would like to see the battalion develop a practice of centrally ordering a lot of the standardized equipment – e.g. Chem Lights – and keeping an inventory on-hand for the ODAs to draw from. It removes paperwork for the ODAs but also makes the ordering and budgeting easier.

• Wanted to know what the system could do to help the XOs, S4, and RMO.  [Laid our the vision of how the system would actually be able to class the colors of money; potentially forecast supplies; track & depict METL; give air officers, ammo officers, and other similar roles heads-up on training events.

• He clarified that since the budgeting cycle happens twice yearly the budget worksheet gets completed each time. That means teams that are in the midst of their training plan are updating the budget to show what has been completed and what they have yet to do.

17. Cpt. Batera

• It would be great if the system could also maintain the historical record of the past completed trainings. This would make it easy for the commander to look back and see what METL items had been hit when it comes time to plan the next training cycle. [Note: Make a report/view depicting how many times and given METL has been hit historically and when.]
• When the Cpt rotates out of the command, he would give/appoint his successor access to the ODAs profile to preserve this historical view.
• Current process is 5 of 5 pain. Lots of weekends.

 

Team 2 Week 4 Blog

 

This week our beneficiary discovery focused on testing our new prototype, which developed significantly from last week.

 

Summary of Key Insights and Trends:

• We are on the right track with the MVP
• Negatives: implementation needs to be consistent across group in order for this to work, this could add more time, B track with input from all members of the team would be redundant, need more flexibility in options (perhaps an “other” category)
• Positives: User interface was good, dropdown menus helpful, a one stop tool would improve longevity and organization of records, would give you costs quickly.

 

1. Cpt Stuart-Shore

• Feedback on this week’s prototype:  “This would be a really useful tool.”
• His ODA teams created their training plans in a team meeting in which they all sit in front of Excel.
• The base level questions the team asks as they plan for the next deployment:
  • What capabilities will be needed?
  • What equipment will be needed?
  • Who will be on the team?

 

2. Captain Dillon

• The mockup is very user friendly! Push notifications are good and save time (2-3 out of 5. It doesn’t make things worse, but isn’t revolutionary for ODA commanders)
• A huge bonus is that it builds information collection into the system, and will address the problem of longevity
• The best aspect is that you can compare to previous years– you now have a system of record that will last as guys rotate out.
• Perhaps this can then plug into the ATN (Army Training Network) or ATRRS (Army Training Resources and Requirement System)

 

3. Senior Medical Sergeant Steven

• 18-series team members would ABSOLUTELY BE WILLING to do extra paperwork if it helps the next team, and if it can be used to argue for more training.
• Everyone on the team besides the 18A is a subject matter expert, and what they say is the 18A’s best estimate of what is needed.
• (As an 18D) We had such little paperwork that I can’t even remember it. It would be burdensome to now have way more.
• Every discipline has a list of 3-5 things that every team member needs to be able to do without supervision. If we know that everyone has their top-5 things, and everyone performs these 2 standards with no supervision, we’re ready. Example: w/in medical, it’s Airway, IV, tourniquet. Perhaps that can be applied in the record keeping to help better determine the $x = Y readiness equation.

 

4. Chief Greg

• The budget part is easy to use, and gives you your overall costs quickly
• It would be really easy for this to be the first part of the process– the Army would then require you to follow up with a threat assessment
• Dropdowns save lots of time

 

5. SFC William

• Ammunition offers a number of both upfront and backend opportunities for forecasting and trend analysis.
• Forecasting weapons maintenance could be a win for the ODAs.
• Providing the ODAs a course catalog as well as Amazon style ratings could be a win for the ODAs.

 

6. IMO Christian

• Getting any type of software approved for SIPR is an extensive, time intensive process.
• Program will be most useful if done in Microsoft Sharepoint or html, as the military is most familiar with those

 

7. Telephone Control Operator Brian

• If you choose to go through with an app, military does use generic cell phones which they integrate into military software. This would be difficult to get approved, like Christian said.
• ODAs use same hardware on base and in the field

 

8. Former ODA and Company Commander Aaron

• MVP is useful if its integrated at battalion AND group, if the systems don’t talk it becomes extra work
• Thinks that we should add an “other” slot to allow the team to add expenses unique to a particular mission or training event
• Prefers the A track- making the whole team give direct input would be redundant and more work

 

9. Major Ryan – Battalion XO

• While IT/Security will be a concern in the near future, nothing about our current MVP locks us into an architectural direction that cannot be accommodated by the Special Forces IT infrastructure.
• Initially focusing on the ODA experience (Product-Market fit) over the other beneficiaries makes sense from a prioritization standpoint.