Customer Interviews (11/21)

 

SGM Angel – Future Concepts

• Combat Development Directorate divided into traditional acquisitions and emergent technologies.
• Traditional acquisition units considered to have healthy acquisition process while emergent tech units struggle.
• R&D leaders don’t necessarily want us to come to them with a list of vendors, which they might perceive as interfering with their process.

 

SGM Brian – R&D Section Leader – “Specialized Capabilities”

• Noted that not all units would value a collaborative tool equally. Considers himself specialized enough that he doesn’t overlap much with anyone else.
• Even so, it would be nice to see if anyone visited a vendor that he plans on talking to.
• We have a process for allowing vendors to submit information over a classified network, so garnering some kind of vendor participation isn’t an impossible prospect.
• When using trip reports on his own team, he would look for the following information: Quotes, Official Messaging, Meeting Notes, Briefing Materials, Technical Data Package, Test Reports, Significant Emails, Overall Capability Wishlist.

 

MSG Darrell – R&D Section Leader – Autonomous/Swarm Vehicles

• There is a core group at the lower level that’s tech savvy and tracking what’s going on with the industry. Problems occur sometimes when we assign individuals who may not be best suited to the role to essentially go out and headhunt, and they may not fully understand the capabilities.
• Being able to track some of our engagements would help mitigate this problem – our “highly motivated introverts” can see what is being sent back and evaluate.
• However, there is a natural competitive nature here, and people want to solve the problem on their own. Not everyone sees the benefit of sharing, some may even see it as a detriment.
• Would find it useful to be able to look up what’s available by vendor and who has contacted a given vendor. Sees a lot of potential problems with including points of contact on such a page.

 

CIV John – RTI – Former Command Science Advisor to USASOC

• The real problem we have, (which we learned from Joe would be illegal at a vendor level but still causes problems on an interagency level) is that an agency like DARPA will talk to one person at Special Ops who requests a particular technology, which we can’t actually afford and which he doesn’t actually have the authority to request.
• We developed the requirements process to discipline that, and the Science and Tech Components Council was meant to alleviate that. We would conduct field tests on 40-50 techs to show off capabilities and provide feedback to developers.
• Our assessments provided a sort of visualized rating that could be quickly digested and give a rough glimpse of a tech’s performance.
• The agency as a whole needs a more unified view of the process.
• No database cuts across classified and unclassified networks.

 

MSG Matt – C4I Commodity Area SME – Comms and Data Management Systems

• It would help to be able to see what a particular person I know to be working on a given problem has done.
• There are three official types of paperwork you might need to file when engaging with a vendor: Request for Orders, travel vouchers, or Visitor Request Process. These essentially let the unit know what you’re doing and allow the unit to reimburse you for travel.
• The trip report memorandum is essentially a follow up to this process and isn’t required for anything. As a result they rarely get done.

 

MSG Joe – C4I Commodity Area SME – Intel and Data Management Systems

• We manage all of our projects in a portfolio but we sometimes look at capabilities not necessarily in our portfolio.
• Our goal is to catalogue and earmark capabilities even if we’re not directly working on it.
• We really need a basic categorization of our capabilities in order to better track how we our meeting our requirements.

 

MSG Tony – C4I Commodity Area SME – Specialized Comms Systems

• I would love to be able to apply keywords to vendors to increase the likelihood that someone interested in my highly specialized techs could find it in a search.
• I also want to avoid piling on a vendor – we can wear them out if a bunch of us visit for the same thing.
• If I could save 2 hours of market research by checking a database, I would consider that a valuable result. None of us have enough hours in the day.

 

MAJ Mike – Commander Action Group Director – Special Air Operations in support of Joint Special Operations Command

• When I’m preparing to meet a vendor I have two major questions: Has there been a previous contact from my or a related military organization and if so, what was discussed and what do you recommend for action?
• At my level I would love to have a more sophisticated way of tracking vendors, almost like the way we map terrorist organizations.
• The only way we can really disseminate info now is socially in things like staff meetings or point to point contacts.

 

MSG Paul – Technical SME – Well rounded SME supporting multiple units

• Two things I’d like to be able to do: quickly know what our guys think of a vendor, and whether we can combine efforts into a single trip to a vendor.
• Do NOT allow anyone to access vendor contacts. First, we don’t want just anyone calling these vendors all the time. Second, people guard their contacts jealously and this could make them reluctant to use the system.
• Another problem that’s harder to solve is what to do when people are rotating out of their positions every 3 or so years? How is their replacement going to know who to talk to? How does the guy leaving know what information will be most important to leave behind?
• Historical trip reports aren’t a direct help, more useful as a catalogue of who has visted who over time.
• If it takes more than 10 seconds to understand, don’t bother.

 

LTC Phil – DARPA Program Manager

• DARPA Program Managers have a lot more independent authority, nobody is really there long enough to overlap or know each other.
• We use Trelo to track ongoing tasks.
• It would be helpful to find ways to collaborate, but we are also concerned about the integrity of our own projects.

 

MSG Ryan – R&D Section Leader – Unmanned UAS

• It’s relatively new for us to actively store trip data. I keep a personal stockpile of trips relevant to me but don’t put it anywhere central.
• When hardware is tested, the testers produce “After Action Reviews,” which may contain more direct performance data than a trip report. However, these also come back to me and I have to hold on to them.
• I personally believe we need to make more use of AARs in particular so we can actually learn from our mistakes.

 

Hypotheses Supported

 

People within CDD and R&D sections would like to know who has visited a vendor they are researching.  Not everyone expressed equal levels of excitement about this, but almost universally people agreed that if such a list were easy to access it would be worth checking to see if they can make any useful contacts within the organization.

 

People within CDD and R&D sections would like to know who plans to go on a trip. This appears to be more important to R&D section leaders, who might be able to save time visiting a vendor if someone else already has a trip planned or perhaps identify areas of operational overlap.

 

Hypotheses Debunked

 

People within CDD and R&D want to know about points of contact at a given vendor. This idea was almost universally discouraged, noting potential issues of vendor fatigue and an existing culture of guarding contacts. Such an inclusion could diminish use of a solution.

People within CDD and R&D want to know what is in the trip reports. Our interviews have forced us to reconsider the importance of trip reports to the majority of people who might use a potential solution. While some individuals are highly interested in using trip reports to coordinate research efforts, many just want to know who to speak with about a given project.

 

The process revolves around trips. Following a lengthy conversation with our sponsor, we had to revise one of our core assumptions: that our main point of focus was getting people to more actively share the details of these trips. Trips are only one component of the larger deployment process, and the way R&D Section Leaders approaches trips directly impacts the way CDD employees plan each year’s budget. Our future lines of questioning will need to dig into this relationship more.

 

Questions for this week:

1. How does the relationship between R&D Leaders and Commodity Area specialists affect which technologies are deployed?
2. What types of miscommunications occur between R&D sections and Commodity Area units?
3. Do R&D section leaders believe that communicating with CDD will increase the likelihood that their techs are deployed?

          US Army Special Operations Command (USASOC) challenged Team 7 to develop a method of tracking external engagements to better coordinate external interactions and increase networking opportunities among Commodity Area Directors. After an introductory meeting with LTC Joe, we sought out interviews with his suggested contacts.

Customers Interviewed

CW4 Jimmy – Tech Targeting Commodity Area Director(CAD)

• Used example of 3D printers. If a CAD meets with a potential vendor to discuss 3D printers, he or she sends that report to whoever they know is also working on 3D printers. An individual outside that person’s direct network has no way of finding the report.
• Expressed a desire for dropdown menus to enforce some standardization of terminology. Wants any standardized trip report form to include who they visited, the time frame, and the topic of discussion.
• Noted lack of standardized trip report form; trip report could be submitted in the form of a Word document, PDF, or Powerpoint presentation.

SGM Katie – Tech Targeting Commodity Area Subject Matter Expert

• Said there is no organized view of vendors. Something as simple as being able to search which vendors have made some kind of contact would improve immediately.
• Expressed a desire to know which offices have been in contact with a given vendor.
• Emphasized cutting down time of data entry. A solution should not require a substantial time investment.

SGM Alex – Tech Targeting Commodity Area SGM

• Most of the redundancy that stems from lack of communication leads to time wasted rather than redundant expenditures.
• Person filing trip report currently faces dilemma of not sending report to those who could benefit and spamming people unnecessarily, which can drive down engagement of trip reports.
• There isn’t necessarily a tangible benefit to the person writing the trip report if someone else finds it useful, other than knowing their trip wasn’t a waste.

Mike  – Command, Control, Communications, Computers, and Intelligence (C4I) Commodity Area Deputy

• Uses a OneNote system that can move reports up the chain of command but program clunkiness, bugs, and trouble functioning as historical record all prevent it from being an ideal solution. Trip reports are included but not updated.
• Also use “purchase orders” as documents they manage within database.
• Doesn’t believe people outside their group are interested in their trip reports. They are shared internally and up the chain of command.
• Also believes his unit is primarily interested in their own trip reports and wouldn’t get much value out of knowing what other units are doing.

MSG Tom – Future Concepts Technical Subject Matter Expert

• MSG Tom tracks what other units are doing and tries to inform them of relevant capabilities, which currently requires point to point connections because there is no defined process or centralized distribution point. He does not have authority over who submits what kind of data.
• Does not believe people will take the time to perform additional data entry. Believes people who go out to find solutions are “problem solvers” first.
• Some standard data points are captured in the process of going on a trip. Trip reports also typically include who they met, what they discussed, the date, and agenda. However, some local meetings particularly in the DC area could lack typical data points.
• Has yet to find any example across USG of successfully cracking this particular problem.

CW3 Jeff – R&D Section Leader

• Working to develop an effective system of information sharing. The “wicked” problem: their ability to collect information rapidly outpaces their ability to process and disseminate.
• Due to the wide variety of trip report formats, particularly PDFs and Powerpoints, it is difficult to extract pertinent points of data and put it in a palatable format.
• The system of emailing trip reports is the biggest problem, such reports can be lost forever in someone’s inbox.
• Has tried Skype and MERCchat to more actively share information.
• Currently exploring the possibility of working with Slack or developing and deploying an in house solution, both are still in the nascent phase.

MAJ Josh – R&D Section Leader

• MAJ Pugh’s unit works primarily on drones.
• Lots of other groups depend on drone tech & would love to see trip reports from this group such as which drones are relevant to a particular capability and which drones don’t meet performance requirements.
• They built a sharepoint database that contains trip reports and other data in hopes to share this knowledge with others. It is brand new and unproven, but could potentially be scaled if it works. However, most people in agency do not know about it and MAJ Pugh is not sure how to get the word out.

LTC Chris – R&D Section Leader

• LTC Chris’s section functions more as a team of “requirements generators and subject matter experts” than focusing on a particular tech sector.
• Particularly interested in engagements across agency, has attempted to coordinate outside interactions via Microsoft Sharepoint. They went so far as to have in house engineers attempt to modify the program to better meet the unit’s needs with no success.
• Believes that trip reports are viewed as administrative work, separate from the “real work” of problem solving. As such, individuals filing trip reports are unlikely to take a lot of extra time to make sure any proposed solution functions as intended.

Seth – R&D Section Member

• Microsoft Outlook’s calendar tool is great for organizing meetings but can’t be seen on sharepoint and can’t be posted publicly so everyone can see.
• While the system is more organized, it still essentially requires point to point contact to work with the added burden of a non-intuitive system.
• Believes any effective system must be quickly responsive; if the system is slow people will give up on it.

MSG Tom Followup

• Follow up to discuss challenges associated with implementing a systemic knowledge management solution.
• Noted that the majority of trip reports start out as Word documents.
• However, when they move between classified and unclassified networks they are printed to a PDF, making it exceptionally difficult to search.
• As a result, one report may have multiple associated digital files.
• A solution must also account for transitions between classified and unclassified networks.

Key Insights

• Not all units treat trip reports the same way. Trip reports have no standardized formats, different units use different systems to track their data, and not all units see value in sharing trip reports across the agency or spending considerable time on them at all.
• Individuals across USASOC have attempted to develop solutions to the problem ranging from leveraging existing Office programs to building an in house program from scratch. Such attempts were typically isolated to the unit of the person promoting it, and individuals who developed solutions consistently expressed the difficulty of spreading the word and building support.
• Groups are often unaware of the activities of other groups, both in terms of having no access to their documents, and even to the extent of having no awareness of their efforts to share those documents.

Hypotheses Supported

• There is no centralized repository for trip reports. Individuals submit trip reports along the guidelines of their unit’s process. Short of combing the entire email system for Office attachments, there is no place to find the entirety of the agency’s trip reports and relevant data.

• Customers will benefit from greater access to trip reports. While this hypothesis did not prove universally true, several of our customers actively attempt to connect deployment efforts as part of their job description and would directly benefit from being able to track trip report data more reliably.

• There may be other types of data beyond trip reports within our scope. The sUAS group mentioned their most recent efforts to catalogue trip reports included other data beyond just trip reports, suggesting we may need to widen the scope of the inquiry. Furthermore, Tom informed us that there are other type of events similar to trips (such as bringing vendors to the base) that generated the same kind of information, but are not being captured in trip reports, suggesting we may have to solve social problems around how the unit captures data to begin with.
• No good solutions to this problem that would be appropriate for deployment here exist. We have more work to do to confirm this, but Tom, for whom solving this problem is one of his central responsibilities, has attested that he has been searching high and low within government agencies for how other groups solve this problem, and has found emphatically no success.

Hypotheses Debunked

• Units within the agency have not spent considerable time developing a solution. Our initial interview with Joe gave us the impression that he attempted one type of solution on his own but that support was not widespread. Our interviews revealed a number of these individualized efforts, many of which also failed to garner widespread attention.

• We can count on individuals to file trip reports. Our interviews revealed a general lack of accountability when it came to tracking who filed a trip. As such, some individuals do not see the value in filing them at all. This countered our initial assumption that trip reports may be hard to track but are reliably submitted when a trip occurs.

• Customers who file trip reports will benefit from having a central repository. While Commodity Area Directors saw an obvious value in being able to track various trip reports, our customers consistently reported that individuals who file trip reports don’t expect to benefit from an expanded network. As such, we cannot assume they will be particularly motivated to perform additional data entry.