Questions for This Week

• MVP Applications: Is our tagging system effectively capturing key engagement details? Is our tagging input system intuitive? What information would beneficiaries submit using our form?
• Unit Dynamic: How can we ensure that R&D does not feel they are taking a major risk submitting their data to our system? What boundaries are necessary for each of our beneficiaries to accomplish their goals in this system? How do we achieve sufficient buy in across the unit to implement the system?
• Dual Use: Can people from other agencies understand the intent of our MVP? Can they envision how they might use it in their context, even if it’s currently built with USASOC’s workflow in mind?

What We Are Testing

• Our manual input form is intuitive and easy to use.
• Our manual input form only asks for relevant information.
• Our manual input form allows for the submission of all relevant information.
• Individuals who are highly concerned about CDD interference will not share any information about their projects.
• If the squadrons and CDD buy in to the project, apathy from higher ups will not be a problem when it comes to implementation.
• Beneficiaries seeking out a project have enough context to work with limited information. [condensing search output]
• Search and input MVPs are relevant to different kinds of workflows.

Key Insights

Our most important feedback this week was that our input MVP seemed to have the flexibility to cover a range of reporting that we did not anticipate. Our contacts in AFSOC and JSOC both cited ways they could see themselves using it despite that the MVP was not designed with their workflow in mind. This makes us more confident that we are striking the right balance between having some operational guidelines while providing considerable flexibility to submit whatever information is relevant. It also suggests that such a program could be functional across units without drastic changes.

Culturally, we learned from Nic in R&D that the core communication issue isn’t necessarily that people’s projects will get shut down. The real issue, in his opinion, is that people don’t feel like they’re being heard when they do talk to CDD and therefore don’t really see an upside to reaching out. Shutting down projects might not be a huge problem, but it’s still a downside, and the downside is what people will think about if they don’t perceive an upside. In theory this perspective fits with some of the prior testimony we have heard, but we would like to hear more about it from those who previously expressed concerns about CDD interference.

Interviews (10/82)

SGM Angel – Future Concepts

• Even among people who suspect CDD will try to shut down their projects, there is usually a level of information they are willing to share. Can’t speak to specifics for every individual.
• Affirmed the value of reaching out to Subject Matter Experts after submitting a proposal/requirement. It can be difficult to know if you have all the relevant information just from looking at a report, so the ability to check in is important.
• People can currently check the status of their pre-validation requirements via DecisionTracker. Most people hate it because it’s such a pain to access through the portal.
• Ideas get triaged all the time, particularly at the Requirements Review Board level. People need to accept that some ideas will not move forward.

Ben – AFSOC

• [on search MVP] It would be nice to somehow stitch together pertinent information from multiple site visits/trip reports so the customer could have a consolidated picture of past convos.
• A streaming comments section would help as well to make updates easy.
• Easy add: recency filter. We should be able to cut out stuff that doesn’t matter anymore.
• Harder add: relevancy filter; “Decision X has been made which renders Y information obsolete.”
• Wants to know how old information gets scrubbed so that the search function stays clean.

Brian – Signal Squadron

• [on input MVP] I was just on a trip that I would need to fill one of these out for.
• Thinks the types of inputs we are requesting would be easily searchable.
• Did not as easily envision the exact process by which the form itself would be submitted.

Doug Brook – Professor, Fmr. Asst Secretary of Navy

• [Inquired about the issue of siloing at the DoD level]
• The Pentagon has uncountable silos.
• Silos can be organizational, functional, or operational.
• As such, it’s difficult to provide an illustrative example of exactly how these divisions play out. Rather, it’s more accurate to say that each individual views whatever topic is under discussion through the narrow lens of their job.
• Leaders should theoretically bridge divide but there are many ways to hamper initiatives that don’t fit your interests.

MSG Matt – C4I

• [on search MVP] “Very [useful], I would likely use this at the outset of every project.”
• When researching, would use MVP to get a specific vendor’s history; any past interactions, any notes on performance or capabilities.
• Would also want to use it to get a broad overview of what we’re doing in a given technology area.
• Helpful deconfliction tool.

MAJ Mike – CAG in support of JSOC

• Suggested we look into Project Vulcan.
• More limited in scope than what we are doing as it mainly solicits information from vendors
• This functionality alone was enough to generate some buzz with his people; he found out about this program by chance through one of his teams.

Nic – Squadron R&D Deployment

• The whole building is visual, you have to be careful about adding too many layers to this thing.
• Doesn’t necessarily care about the timeline of when his requirements get validated. Just wants to know when the tech he wants will be in his hands.
• One element of the CDD/R&D issue is trust. People feel like they’re not heard when they do speak up, so there isn’t a lot of upside to sharing. Easier to think about the negatives.
• End users need to be able to focus on the mission. We either need to add non-operational people to the squadrons to focus on this stuff or have CDD do it.

Rob – CDD Operations

• Currently uses a computer system to submit RFOs.
• Would not intuitively know enough about his colleagues trips to fill in the gaps if a trip report wasn’t clear on, for example, whether they were visiting a particular vendor.
• All vendor related trips can be shared organization-wide. He does not suspect anyone on his team would object if he decided to share some of their reports.

CAPT Nick – Kessel Run

• Air Force going through similar issues as Army. Will deploy products 3-5 years after everyone stopped asking for them.
• Decisionmaking in the military is most frequently top down and this causes problems. Higher ups make decisions based on their experiences, which can be miles removed from the situation on the ground today.
• Believes that buy in from higher ups is still necessary but can be attained if you show the value the program has for subordinates. Everyone wants to look like they’re doing a good job, so is there a way that supporting the program will help him do that?

MAJ Mike – CAG in support of JSOC (MVP feedback)

• Liked the balance of available tagging options and flexibility to add new ones in input MVP.
• Would expect to see all engagements go through this system.
• Key info: who, what, where, when, why, and how to proceed.
• Also helps to know who set it up, the original purpose of the meeting, and whether the outcome aligns with the original purpose.
• Would be nice to be able to search the content of the trip reports.

Hypotheses Confirmed/Debunked

• Confirmed: Our manual input form is intuitive and easy to use. Most users seemed to understand the purpose of each tag.
• Partially Confirmed: Our manual input form only asks for relevant information. Nobody pointed out a particular tag as being irrelevant. However, nobody would use every tag on the list every time, so it’s hard to speak to what the unit as a whole would find useful.
• Partially Confirmed: Our manual input form allows for the submission of all relevant information. It seems the flexibility of the system empowers people to think about how they’d submit whatever form they’re thinking about. Our Air Force contacts discussed using it to submit forms that we were unaware of.
• Debunked: Individuals who are highly concerned about CDD interference will not share any information about their projects. There is some division as to exactly why some people are so worried, but this may be tied to the amount of information shared rather than whether it’s shared at all.
• Partially Confirmed: If the squadrons and CDD buy in to the project, apathy from higher ups will not be a problem when it comes to implementation. We will not know until we actually try. However, interviewees indicated the value of buy in from the ground up when getting a higher up’s approval.
• Debunked: Beneficiaries seeking out a project have enough context to work with limited information. [condensing search output] We found instances where people wouldn’t know things we would expect them to contextually know. However, people are also OK with getting a good contact an reaching out.
• Confirmed: Search and input MVPs are relevant to different kinds of workflows. Its relevance to our contacts from outside USASOC indicates it has a degree of flexibility.

Questions for Next Week

Is our input system more convenient than whatever people are doing now?

How many forms could our input system replace?

What amount of limited information is even the most paranoid R&D employee willing to share freely?

How can the search interface be condensed effectively? What kind of info should a condensed version display?

What value can CDD deliver that makes communicating with them worth the risk of getting a project shut down?

How can supporting our product benefit higher ranking people at USASOC, even if they don’t intend to use it?