Team 7 Week 3 Progress Report
Interviews (17/42, 21 total for weeks 2 and 3)
Tim (Salesforce)
- Believes users will enter data if they believe it will make their life easier.
- Thinks unified adoption will require strong top down leadership.
- Believes it is advantageous when as much data remains unclassified as possible.
- Salesforce has some experience working with other SOCOM divisions but has not fully completed FedRAMP.
Natalya (Salesforce)
- Cost of this type of program is open ended; usually requires modification and government cloud is more expensive.
- Uses iterative model involving weekly feedback from end users.
John (DARPA)
- DARPA experiences similar info management problems.
- He is constantly approached by vendors offering new knowledge management features.
- Different groups will have different tolerance levels for doing work to make your solution function but are unlikely to want to do more than they currently do. The F-35 team for example is accustomed to a large paper trail.
- Solution should function with the organization as it currently exists and not depend on the organization changing.
General Dempsey
- Money prevents collaboration. People want funding for their own projects.
- However, this isn’t necessarily a bad thing. Friction between these groups is what moves the most viable projects forward.
- Friction is intensified when budget is tight.
Fort Bragg Interviews (ranks need to be confirmed, units are correct)
Rob – CDD
- Biggest pain point – finding out about an unexpected procurement after he’s already completed his spend plan. It’s very hard to change things once the plan is submitted.
- Believes part of the color of money reporting problem is the parameters around each kind. CDD should get a notice for any O&M purchase.
- We need to simplify the rules in a way that delivers our core informational needs.
Tom – Future Concepts
- We want to plan for when capability development is successful. If they fail it’s a waste of money but when it’s a success it’s almost worse because we have to decide to cut other things.
- When that kind of conflict occurs we have to go through a Requirements Review Board process that delays deployment.
- We need to be able to ask people whether they want those sudden requests or the $X worth of things we’re going to have to drop because of it, whatever’s at the bottom of the priority list.
Dave – A Squadron
- When I recognize a problem, I want to know my parameters. I want to know who has already looked at it, get a basis for where to start.
- Every squadron produces SITREFS. How can I get a notification when one comes along that is relevant to me?
- When I’m looking, I want to know who to talk to. Why didn’t it go forward last time? Do I need to revalidate?
Joe – C4I
- We want to protect the unique authorities of the unit command. We don’t want to get in the way of what end users or doing but it’s necessary to protect that authority.
- We also want better efficiency in project management. The effort it takes to find all of these records could be improved considerably.
- We also want to be able to sort by commodity area.
Herb – CDD
- Who is the vendor/organization?
- Lead point of contact in unit
- market research
- Was conversation in person or remote?
John – CDD
- I’ve been here since 1995 and at that time we filed trip reports as a matter of discipline.
- It fell out of practice because people couldn’t really put them to use.
- To get people to submit them we need an efficient way to do it, a place to store it, and a way to access it for all users.
Tom – Future Concepts
- It’s good to save time but we also need buy in from leadership.
- Those trip reports aren’t for me, they’re for whoever is going to read them. So if I don’t think anyone is going to read them, why would I write them?
- I have no preference to exactly what form this system takes, I just think there needs to be a standard.
Brian – B Squadron
- I would argue that the strength of the organization is having a sandbox with parameters.
- Overstandardization could undermine why we operate this way in the first place.
- End users like giving feedback, they don’t like having to go to a meeting to do it.
Angel – Future Concepts
- Part of the problem is lack of planning – how much HW are you doing before you go on a trip?
- Even without a database the organizational knowledge exists.
- People know where to go if they want to talk to an expert, they just don’t do it.
Jason – Squadron ?
- I love tech solutions but I think this is partially a people problem.
- Like a squadron might need a bunch of censors and not even consider talking to signal squadron.
- They also don’t talk to CDD because they think it will slow them down.
Daryl – Intel(?) Squadron – End User
- We’re a bunch of highly motivated introverts. We have to get into the habit of talking to each other.
- If we don’t feel like CDD’s subject matter expert is helpful we’ll go around them.
- It would be great to subscribe to a feed that alerted me on upcoming trips, etc.
Pat – Signal(?) Squadron – End User
- I don’t always know who is going to be interested in what I have to say.
- I’ll send an email to Daryl and later in the day it’s blown out to 30+ people.
- If those people can find a way to request that information from me then they can get it even if I’m not aware of them.
Tony – C4I
- End users will do some work if it’s easier than what they’re doing right now.
- If it’s harder than email don’t even bother.
- Greater challenge will be connecting disparate silos of data.
Hypotheses Confirmed or Debunked
- Confirmed: Late notice of a project set to be deployed affects CDD’s ability to deploy its budget effectively. Late procurements may force CDD to defund other priorities or take the matter to a review board, which delays deployment for both parties.
- Confirmed: “Unofficial” knowledge of an ongoing project gives CDD a chance to plan its budget accordingly. CDD is not guaranteed to find the money needed but has more time to look for funding from its own lines or from outside agencies spending for a similar purpose.
- Confirmed: R&D leaders see some risk in interacting with CDD. While the sentiment was not universal, there is some perception that they will be slowed down or denied.
- Debunked: Trip reports are the standard means of reporting progress. We heard many references to SITREFS, with a couple even citing it as the reason they might find out about a given project before they are told.
- Debunked: End users aren’t interested in providing feedback. They are interested but hate having to go to a meeting to do it. As such, only those who really want to see a capability deployed will show up. More simplified feedback systems have worked better.
Questions for Next Week:
- Can we build a timeline out of our paper trail? Can we make reliable assumptions about the order in which our milestones would appear?
- Could certain O&M transactions be detected that would forecast a potential future procurement, and would CDD be able to parse such information effectively?
- To what extent to individual teams rely on their own way of doing things? Would they prefer a standardized system? What are they not willing to give up under a new system?
Team 7 Progress Report – Week 2 (Base Visit Week)
This week is base visit week and given our discoveries about the size of the workflow last week it has given us an opportunity to make sure we truly understand the organization, the processes they are concerned about, and key differences between the various groups that fill similar functions in different commodity areas.
Customer Interviews (4/25)
CW3 Steve – Network Operations
- Organization is currently working on boosting cloud offerings, including an unclassified cloud that could facilitate collaboration
- However, there is no organizational standard on where or how to keep data, which may not be fixable.
- We do much more work on the classified network because it is easier to share internally and there are more capabilities.
- It’s possible to get a vendor’s info on a classified network but everyone has a different process.
MGSGT Chad – Squadron Ops SGM – Funding for In-Unit R&D
- Not embedded in R&D section but supervises R&D section in Squadron F. Coordinates all of F’s activities and maintains continuity between projects.
- We have a unique mission set and receive special funding from Congress as a result. We can use this budget to tell CDD what we want.
- Each squadron has its own money.
- Individuals highly specialized and can’t necessarily cover each other’s fields effectively.
Ben Fulton – AFSOC
- AFWERX has a better view of vendor databases and which are worthwhile.
- Enables you to see what other Air Force Units are submitting similar problem statements.
- Sees some policies and regulations as getting in the way of acquisitions, but thinks it can be worked around.
- Pointed to an entrenched mindset of “Rapid innovation threatens job security.”
Tom Williams – Commodity Area SME – Communications
- Our job as commodity area SMEs is as enablers of capability development. They look at mission needs, capability requirements, and derived requirements and take the development process all the way to the point of obligation of government funds.
- Wishes end users would understand that taking a little extra time to go through CDD could allow for so many more resources and will ensure that those requirements are sustainable. Suggested that some users take dangerous shortcuts to cut a small amount of time away from the cycle.
- Believes some kind of focal point for end users where they could be put in contact with the right collaborator could be helpful, believes the benefit would be for the end users rather than CDD.
- As a general rule, believes CDD to be doing quite well (particularly given his experience in the Navy), and believes that the key is better communication with end users and convincing end users that taking a little bit of extra time to do procurement properly will be better for them in the long run.
Hypotheses Supported:
- R&D Sections are in charge of early, “unofficial” testing, CDD takes the requirements generated from that stage and eventually brings them to deployment. Our revised hypothesis on organizational structure remains intact following this round of interviews and it would appear we are on the right track of capturing who does what.
- R&D sections have flexibility to act quickly; CDD relies on requirements to act but has more legal authority to enable official R&D and procurement. Our hypothesis on the ‘characterization’ of these groups continues to hold water as well. Of particular note is fairly consistent testimony from CDD personnel that if end users could just be patient and not try to circumvent the acquisition process, there wouldn’t be nearly as much to worry about.
Hypotheses Debunked
- Getting information onto classified networks makes it unreadable by a program. Our interviews with some of the database experts at the organization revealed a number of methods for safely getting information on a classified network, including a system that allows vendors to send information directly. The problem is that most people don’t use them.
Questions for the Coming Week
- Why isn’t the CDD process perceived as fast enough? Are there any examples of CDD slowness compromising a mission?
- Who among the end user squadrons makes the decision to circumvent CDD to obtain a product? Is this shortcut only possible with O&M money and the micropurchase threshold or are there other ways?
- Is there any level of distinction between CDD’s yearly budgeting and its process of deploying tech to meet a particular requirement? (i.e. What exactly do they achieve in the budgeting process and does it give them the leeway to obligate funds toward some new requirement during that year?)
- Do we have the workflows and org chart right? (During our base visit we will be sharing some workflows and our organizational chart with our interviewees to give them an opportunity to correct any misconceptions we have.)
Team 7 Progress Report – Week 1
Customer Interviews (11/21)
SGM Angel – Future Concepts
- Combat Development Directorate divided into traditional acquisitions and emergent technologies.
- Traditional acquisition units considered to have healthy acquisition process while emergent tech units struggle.
- R&D leaders don’t necessarily want us to come to them with a list of vendors, which they might perceive as interfering with their process.
SGM Brian – R&D Section Leader – “Specialized Capabilities”
- Noted that not all units would value a collaborative tool equally. Considers himself specialized enough that he doesn’t overlap much with anyone else.
- Even so, it would be nice to see if anyone visited a vendor that he plans on talking to.
- We have a process for allowing vendors to submit information over a classified network, so garnering some kind of vendor participation isn’t an impossible prospect.
- When using trip reports on his own team, he would look for the following information: Quotes, Official Messaging, Meeting Notes, Briefing Materials, Technical Data Package, Test Reports, Significant Emails, Overall Capability Wishlist.
MSG Darrell – R&D Section Leader – Autonomous/Swarm Vehicles
- There is a core group at the lower level that’s tech savvy and tracking what’s going on with the industry. Problems occur sometimes when we assign individuals who may not be best suited to the role to essentially go out and headhunt, and they may not fully understand the capabilities.
- Being able to track some of our engagements would help mitigate this problem – our “highly motivated introverts” can see what is being sent back and evaluate.
- However, there is a natural competitive nature here, and people want to solve the problem on their own. Not everyone sees the benefit of sharing, some may even see it as a detriment.
- Would find it useful to be able to look up what’s available by vendor and who has contacted a given vendor. Sees a lot of potential problems with including points of contact on such a page.
CIV John – RTI – Former Command Science Advisor to USASOC
- The real problem we have, (which we learned from Joe would be illegal at a vendor level but still causes problems on an interagency level) is that an agency like DARPA will talk to one person at Special Ops who requests a particular technology, which we can’t actually afford and which he doesn’t actually have the authority to request.
- We developed the requirements process to discipline that, and the Science and Tech Components Council was meant to alleviate that. We would conduct field tests on 40-50 techs to show off capabilities and provide feedback to developers.
- Our assessments provided a sort of visualized rating that could be quickly digested and give a rough glimpse of a tech’s performance.
- The agency as a whole needs a more unified view of the process.
- No database cuts across classified and unclassified networks.
MSG Matt – C4I Commodity Area SME – Comms and Data Management Systems
- It would help to be able to see what a particular person I know to be working on a given problem has done.
- There are three official types of paperwork you might need to file when engaging with a vendor: Request for Orders, travel vouchers, or Visitor Request Process. These essentially let the unit know what you’re doing and allow the unit to reimburse you for travel.
- The trip report memorandum is essentially a follow up to this process and isn’t required for anything. As a result they rarely get done.
MSG Joe – C4I Commodity Area SME – Intel and Data Management Systems
- We manage all of our projects in a portfolio but we sometimes look at capabilities not necessarily in our portfolio.
- Our goal is to catalogue and earmark capabilities even if we’re not directly working on it.
- We really need a basic categorization of our capabilities in order to better track how we our meeting our requirements.
MSG Tony – C4I Commodity Area SME – Specialized Comms Systems
- I would love to be able to apply keywords to vendors to increase the likelihood that someone interested in my highly specialized techs could find it in a search.
- I also want to avoid piling on a vendor – we can wear them out if a bunch of us visit for the same thing.
- If I could save 2 hours of market research by checking a database, I would consider that a valuable result. None of us have enough hours in the day.
MAJ Mike – Commander Action Group Director – Special Air Operations in support of Joint Special Operations Command
- When I’m preparing to meet a vendor I have two major questions: Has there been a previous contact from my or a related military organization and if so, what was discussed and what do you recommend for action?
- At my level I would love to have a more sophisticated way of tracking vendors, almost like the way we map terrorist organizations.
- The only way we can really disseminate info now is socially in things like staff meetings or point to point contacts.
MSG Paul – Technical SME – Well rounded SME supporting multiple units
- Two things I’d like to be able to do: quickly know what our guys think of a vendor, and whether we can combine efforts into a single trip to a vendor.
- Do NOT allow anyone to access vendor contacts. First, we don’t want just anyone calling these vendors all the time. Second, people guard their contacts jealously and this could make them reluctant to use the system.
- Another problem that’s harder to solve is what to do when people are rotating out of their positions every 3 or so years? How is their replacement going to know who to talk to? How does the guy leaving know what information will be most important to leave behind?
- Historical trip reports aren’t a direct help, more useful as a catalogue of who has visted who over time.
- If it takes more than 10 seconds to understand, don’t bother.
LTC Phil – DARPA Program Manager
- DARPA Program Managers have a lot more independent authority, nobody is really there long enough to overlap or know each other.
- We use Trelo to track ongoing tasks.
- It would be helpful to find ways to collaborate, but we are also concerned about the integrity of our own projects.
MSG Ryan – R&D Section Leader – Unmanned UAS
- It’s relatively new for us to actively store trip data. I keep a personal stockpile of trips relevant to me but don’t put it anywhere central.
- When hardware is tested, the testers produce “After Action Reviews,” which may contain more direct performance data than a trip report. However, these also come back to me and I have to hold on to them.
- I personally believe we need to make more use of AARs in particular so we can actually learn from our mistakes.
Hypotheses Supported
People within CDD and R&D sections would like to know who has visited a vendor they are researching. Not everyone expressed equal levels of excitement about this, but almost universally people agreed that if such a list were easy to access it would be worth checking to see if they can make any useful contacts within the organization.
People within CDD and R&D sections would like to know who plans to go on a trip. This appears to be more important to R&D section leaders, who might be able to save time visiting a vendor if someone else already has a trip planned or perhaps identify areas of operational overlap.
Hypotheses Debunked
People within CDD and R&D want to know about points of contact at a given vendor. This idea was almost universally discouraged, noting potential issues of vendor fatigue and an existing culture of guarding contacts. Such an inclusion could diminish use of a solution.
People within CDD and R&D want to know what is in the trip reports. Our interviews have forced us to reconsider the importance of trip reports to the majority of people who might use a potential solution. While some individuals are highly interested in using trip reports to coordinate research efforts, many just want to know who to speak with about a given project.
The process revolves around trips. Following a lengthy conversation with our sponsor, we had to revise one of our core assumptions: that our main point of focus was getting people to more actively share the details of these trips. Trips are only one component of the larger deployment process, and the way R&D Section Leaders approaches trips directly impacts the way CDD employees plan each year’s budget. Our future lines of questioning will need to dig into this relationship more.
Questions for this week:
- How does the relationship between R&D Leaders and Commodity Area specialists affect which technologies are deployed?
- What types of miscommunications occur between R&D sections and Commodity Area units?
- Do R&D section leaders believe that communicating with CDD will increase the likelihood that their techs are deployed?
Team 7 Progress Report – Week 0
US Army Special Operations Command (USASOC) challenged Team 7 to develop a method of tracking external engagements to better coordinate external interactions and increase networking opportunities among Commodity Area Directors. After an introductory meeting with LTC Joe, we sought out interviews with his suggested contacts.
Customers Interviewed
CW4 Jimmy – Tech Targeting Commodity Area Director(CAD)
- Used example of 3D printers. If a CAD meets with a potential vendor to discuss 3D printers, he or she sends that report to whoever they know is also working on 3D printers. An individual outside that person’s direct network has no way of finding the report.
- Expressed a desire for dropdown menus to enforce some standardization of terminology. Wants any standardized trip report form to include who they visited, the time frame, and the topic of discussion.
- Noted lack of standardized trip report form; trip report could be submitted in the form of a Word document, PDF, or Powerpoint presentation.
SGM Katie – Tech Targeting Commodity Area Subject Matter Expert
- Said there is no organized view of vendors. Something as simple as being able to search which vendors have made some kind of contact would improve immediately.
- Expressed a desire to know which offices have been in contact with a given vendor.
- Emphasized cutting down time of data entry. A solution should not require a substantial time investment.
SGM Alex – Tech Targeting Commodity Area SGM
- Most of the redundancy that stems from lack of communication leads to time wasted rather than redundant expenditures.
- Person filing trip report currently faces dilemma of not sending report to those who could benefit and spamming people unnecessarily, which can drive down engagement of trip reports.
- There isn’t necessarily a tangible benefit to the person writing the trip report if someone else finds it useful, other than knowing their trip wasn’t a waste.
Mike – Command, Control, Communications, Computers, and Intelligence (C4I) Commodity Area Deputy
- Uses a OneNote system that can move reports up the chain of command but program clunkiness, bugs, and trouble functioning as historical record all prevent it from being an ideal solution. Trip reports are included but not updated.
- Also use “purchase orders” as documents they manage within database.
- Doesn’t believe people outside their group are interested in their trip reports. They are shared internally and up the chain of command.
- Also believes his unit is primarily interested in their own trip reports and wouldn’t get much value out of knowing what other units are doing.
MSG Tom – Future Concepts Technical Subject Matter Expert
- MSG Tom tracks what other units are doing and tries to inform them of relevant capabilities, which currently requires point to point connections because there is no defined process or centralized distribution point. He does not have authority over who submits what kind of data.
- Does not believe people will take the time to perform additional data entry. Believes people who go out to find solutions are “problem solvers” first.
- Some standard data points are captured in the process of going on a trip. Trip reports also typically include who they met, what they discussed, the date, and agenda. However, some local meetings particularly in the DC area could lack typical data points.
- Has yet to find any example across USG of successfully cracking this particular problem.
CW3 Jeff – R&D Section Leader
- Working to develop an effective system of information sharing. The “wicked” problem: their ability to collect information rapidly outpaces their ability to process and disseminate.
- Due to the wide variety of trip report formats, particularly PDFs and Powerpoints, it is difficult to extract pertinent points of data and put it in a palatable format.
- The system of emailing trip reports is the biggest problem, such reports can be lost forever in someone’s inbox.
- Has tried Skype and MERCchat to more actively share information.
- Currently exploring the possibility of working with Slack or developing and deploying an in house solution, both are still in the nascent phase.
MAJ Josh – R&D Section Leader
- MAJ Pugh’s unit works primarily on drones.
- Lots of other groups depend on drone tech & would love to see trip reports from this group such as which drones are relevant to a particular capability and which drones don’t meet performance requirements.
- They built a sharepoint database that contains trip reports and other data in hopes to share this knowledge with others. It is brand new and unproven, but could potentially be scaled if it works. However, most people in agency do not know about it and MAJ Pugh is not sure how to get the word out.
LTC Chris – R&D Section Leader
- LTC Chris’s section functions more as a team of “requirements generators and subject matter experts” than focusing on a particular tech sector.
- Particularly interested in engagements across agency, has attempted to coordinate outside interactions via Microsoft Sharepoint. They went so far as to have in house engineers attempt to modify the program to better meet the unit’s needs with no success.
- Believes that trip reports are viewed as administrative work, separate from the “real work” of problem solving. As such, individuals filing trip reports are unlikely to take a lot of extra time to make sure any proposed solution functions as intended.
Seth – R&D Section Member
- Microsoft Outlook’s calendar tool is great for organizing meetings but can’t be seen on sharepoint and can’t be posted publicly so everyone can see.
- While the system is more organized, it still essentially requires point to point contact to work with the added burden of a non-intuitive system.
- Believes any effective system must be quickly responsive; if the system is slow people will give up on it.
MSG Tom Followup
- Follow up to discuss challenges associated with implementing a systemic knowledge management solution.
- Noted that the majority of trip reports start out as Word documents.
- However, when they move between classified and unclassified networks they are printed to a PDF, making it exceptionally difficult to search.
- As a result, one report may have multiple associated digital files.
- A solution must also account for transitions between classified and unclassified networks.
Key Insights
- Not all units treat trip reports the same way. Trip reports have no standardized formats, different units use different systems to track their data, and not all units see value in sharing trip reports across the agency or spending considerable time on them at all.
- Individuals across USASOC have attempted to develop solutions to the problem ranging from leveraging existing Office programs to building an in house program from scratch. Such attempts were typically isolated to the unit of the person promoting it, and individuals who developed solutions consistently expressed the difficulty of spreading the word and building support.
- Groups are often unaware of the activities of other groups, both in terms of having no access to their documents, and even to the extent of having no awareness of their efforts to share those documents.
Hypotheses Supported
- There is no centralized repository for trip reports. Individuals submit trip reports along the guidelines of their unit’s process. Short of combing the entire email system for Office attachments, there is no place to find the entirety of the agency’s trip reports and relevant data.
- Customers will benefit from greater access to trip reports. While this hypothesis did not prove universally true, several of our customers actively attempt to connect deployment efforts as part of their job description and would directly benefit from being able to track trip report data more reliably.
- There may be other types of data beyond trip reports within our scope. The sUAS group mentioned their most recent efforts to catalogue trip reports included other data beyond just trip reports, suggesting we may need to widen the scope of the inquiry. Furthermore, Tom informed us that there are other type of events similar to trips (such as bringing vendors to the base) that generated the same kind of information, but are not being captured in trip reports, suggesting we may have to solve social problems around how the unit captures data to begin with.
- No good solutions to this problem that would be appropriate for deployment here exist. We have more work to do to confirm this, but Tom, for whom solving this problem is one of his central responsibilities, has attested that he has been searching high and low within government agencies for how other groups solve this problem, and has found emphatically no success.
Hypotheses Debunked
- Units within the agency have not spent considerable time developing a solution. Our initial interview with Joe gave us the impression that he attempted one type of solution on his own but that support was not widespread. Our interviews revealed a number of these individualized efforts, many of which also failed to garner widespread attention.
- We can count on individuals to file trip reports. Our interviews revealed a general lack of accountability when it came to tracking who filed a trip. As such, some individuals do not see the value in filing them at all. This countered our initial assumption that trip reports may be hard to track but are reliably submitted when a trip occurs.
- Customers who file trip reports will benefit from having a central repository. While Commodity Area Directors saw an obvious value in being able to track various trip reports, our customers consistently reported that individuals who file trip reports don’t expect to benefit from an expanded network. As such, we cannot assume they will be particularly motivated to perform additional data entry.
Recent Comments