• Agrees that organizing by classes provides a better picture than cohorts b/c people who drop out of certain cohorts and then are categorized to a different cohort
• For the normal distribution chart, Phil is interested in the specific characteristics that differentiate the top 5% v. the top 50% and lowest 5%?
• “Cohort Comparison Chart” could include – how do my cohorts compare w/ CG John M, CG X, CG Y? This provides a comparison to CG’s how they are doing to past CG’s

bmm@cs.duke.edu

• DARPA is interested in blockchain for storing soldier demographic information
• DARPA wants additional research or case studies of how blockchain could work to store information and has convened blockchain researchers to learn more

michaeljelen@gmail.com

• Try combining the macro and micro-level data so that only the most important information is present.

adambeauregard@gmail.com

• Everything looks fine.
• Doubtful that the team CPC will go through 12 pages of graphs. Pick out the most important ones.
• Consider making a dashboard.

mheath17@gmail.com

• Try figuring out which graphs are the most important.
• Everything seems aesthetically good.
• Try also to find filtered categories.
• Willing to invite people to the location to meet the team.
• Wants us to send info about H4D demo day on April 18th.

joeblanton12@gmail.com

• The ability for the platform to tailor to the preference of different commander is very important

• Present the so-what and the purpose before explaining the why; assuming a busy and inpatient commander only has less than a minute of time

• An algorithm that take in key indicator to show the changes of weekly key result

• Have evidence and data to back-up what is the drive of attrition

• Regression and projection would help a lot

Factors used to test level of engagement and emotion change:

• Galvanic skin response

• Position shift in alpha brainwave
• Movement synchrony

phillip.thomas2@socom.mil

• Agrees that trainees don’t need access. Reports can be generated for them
• Make sure our efforts are coordinated with Oscar
• Still concerns over data storage

trevor.obrien@socom.mil

• Comparison timeline. Weekly is good, but monthly or even quarterly could be useful for visualization.
• General feedback, good compared it to spear interface.
• Maybe too many tabs at the moment. Navigation may get too complicated.
• Interested in looking at the tactical training.

• Mental health and athletic performance are not separated – he works with both aspects of student athletes to improve sports performance
• Athletes, mostly male, do not trust him, because they have been socialized that showing emotion is weak
  • He spends time at their practices to build trust

• The focus on holistic performance is b/c the pressure and psychological stress that was put on the trainee
  • Soldiers who were going to therapy didn’t want people to know they were going to therapy so that is why there is holistic focus
• Compare soldier performance w/ marine or navy corp people – what information can you correlate among different branches (pushups, metabolic rate)?
• The platform addresses two things they need 1) real-time data 2) cadre v. cadre & soldier v. soldier & military v. navy v. air force

oscar.gonzalez@usuhs.edu

• Holistic Performance – Rader Chart need to have both
  • Physical domain
  • Psychological domain
• Exercise & Performance – what other factor contribute to performance difference?
  • Diet
• Percentage representation is vague, need to specify what’s the unit time
• Need to clearly label each graph and each measurement need to have a unit
• Need to define what ideal means
• The baseline graph shouldn’t be a continue graph – > use bar chart instead
• The position of the sliding bar is confusing

constance.garcia.ctr@socom.mil

• Heart rate variability monitors & eye tracking are the only ones that CPC’s use right now
• Surveys are handed out to the entire cohort
• Surveys change year from year – data analysts can upload surveys into the platform so they can compare whether surveys are comparable

• We don’t want to target specific instructors graduation rates bc it sets up competitive field between instructors
• For CGs, we want to  create four quadrants – CPC, Cadre, physical, academic/survey data
  • CG’s don’t need to know information on specific individuals

aspen.ankney.ctr@socom.mil

• Currently 6 cohorts in a year, will change to 4 cohort next year
• CPC don’t care that much about Cohort performance and organize information by cohort won’t help them, what they want is organize by course(However research psychologist really want the information to be organized by cohort, this is what Oscar care about) — All depends on who is the target user
• Cohort sorting would be the least use feature since few soldiers make through the entire pipeline still stay within his/her original cohort
  • Organize by course make much more sense
• Want data visualization for each course
• Want comparison among different pipeline: Green Beret, Civil Affairs, Psychological Operation
• There are total of 12 CPC on the team, each coach between 80 – 300 students
• Based on Aspen, CPC don’t care about individual profile and overall they are not interested in individual performance that much, they care about course
  • Not that frequent meet with individual students (sometimes not at all, sometimes 10-20 per week, usually only meet with the students who are struggling and those who outperformed)
  • Instead of displaying individual performance they want course performance
• 7 phases, one phase will broke into many sub-courses. Ex: there are five courses within officer specific training
• Based on Aspen, data team and research psychologist don’t have the training and don’t know how to run the analysis that they want —> CPC input data and run analysis themselves —> the product should allow them to input data, organize data, output result and run analysis
• Whatever information accessible to research psychologist should also be made accessible to CPC
• Overall, Aspen loves the prototype and she gave very positive feedback: incredible, easy-to-use, highly developed, awesome job

Daniel.gajewski@socom.mil

• Visuals are good. Good incorporation of feedback from last conversation
• Showing that we are able to take multiple files from soldiers and integrate them into aggregate data is promising.
• Ideally, based on the visualization we have shown, if we have say 8 soldiers in a particular cohort and 7 exercises, this would mean using 56 files to look at holistic and aggregate performance (for simplicity, there would likely be more than one file per training exercise for one soldier).

mark.monturo@gmail.com

• Holistic Performance
  • Leadership ability
  • Physical fitness test score
  • Ability to learn
  • Tactical proficiency
  • Deploymacy
  • Problem solving
• What is the standard of “idea”? In navy they have NSS-navy grading score
  • Put something in in terms: historical score (minimal score, average score, reference line)
  • Standard deviation
• What want to see on the commander’s page:
  • Peer evaluation
  • Student’s hobby and what activities do they do in their spare time
    • Looking for endurance athletes and team player; attract intelligent and all-rounded people
  • Answers to questions such as: Am I producing the next generation? Do I have enough people? Do I need a policy change

seth.c.spradley.ctr@socom.mil

• Says visually it looks good, interface is intuitive.
• Offered to help debug the software system when we develop it further.
• Progression tab, overlay box should be moved to upper left hand corner rather than in the middle of the graph.

mjarth@gmail.com

• Ask if there is an operational psychologist
• Have several “mini-traits”
• Layout the overall situation. Tell your sponsor to pick a specific problem to go after. Can’t solve all the military problems.

dbuck22@gmail.com

• Pitch to companies that have a similar solution and try to get buy-in from them.
• Get rid of red text from the cohort home page. Red means bad in the military
• It would be more ideal to show a passing rate for each class.

phillip.thomas2@socom.mil

• Briefly viewed prototype but had computer issues
• Acquisition process for our product may not be too difficult if the product is relatively inexpensive, could likely be purchased through a simple contract/credit card.
• Issue is, who has the money? LTC Thomas would likely have to lobby either SWCS, USASOC or SOCOM. Process could take months to years depending on politics and who has the money.

• These are the questions we should be asking:
  1. “What is the one thing you want if you could see w/in 20 seconds?”
  2. “What do you think is the most important takeaway from this page?”
  3. “If you were presenting a page for your boss, then what would you include in there?”
• Rank list of priorities for the person to minimize having too many features

• Compare trainees to the “golden references”
• Need golden references for gender, age group – we need to create golden references by other groups
• Figure out average performance of high performers

morgan.hall.ctr@socom.mil

• “It’s all about progression – what is the baseline is and how do I progress on these various indices from that baseline?”
• Operational GB who have been here many years and have had many deployments would be the golden reference
• Would use it most likely when he is having a one on one with the soldier to see which areas they think they’re struggling in

constance.garcia.ctr@socom.mil

• Put the following into soldier info: age, rank, years of service, deployment & how many years deployed, highest education, prior & current MOS (military occupational specialty)
• LOVES: holistic performance overview (spider web graph) – she loves it b/c it shows how each part plays into the other
• Don’t use personal ID information b/c they don’t want this to go on their record – take out “John A.”

travis.nicks@duke.edu

• Based on his 10 years experience working on technology innovation at Navy, it’s very hard to get devices approved to connect to the internet due to risks of cybersecurity threat
  • Even get approved, there will be many constraints on the use case
• Metrics can be used to measure anxiety level besides hrv includes:
  • Breathing rate
  • Sweat
  • Saliva – hydration level
  • Stand still -motion range

barbara.j.plotkin@socom.mil

• We need to be more hands off during captain’s career course training
• Mission analysis is always taken in the form of a problem statement
• Funding begins in October, 4 cycles of CCC, then ends on the end of September
• Comptroller looks at historical data and decides if the needs of CCC are being met with the current level of funding.

seth.c.spradley.ctr@socom.mil

• Prototype holds up well visually. Graph presentation is very effective.
• Interested in seeing more of the achievement history in the soldier performance tab
• Was confused if we were solely focussing on Green Berets or are also including civil affairs and psyops. (regardless, this system should be able to scale to meet their needs)

• “There has been a lot of people pushing for this information, but it has been hard to get units to share information” (about developing golden references)
• The quality of candidates that we were getting almost twenty years ago is remarkably different than the quality of candidates that we are getting today – seventeen years ago, 9/11 happened so we had wall street bankers, Master’s degrees candidates so the quality of education that people are coming in with are very different.
• Visually represent in an intuitive large amounts of data – CPC’s need a platform that brief them on data in an intuitive and self explanatory way

gabriella.shull@duke.edu

• Visual attention EEG
• Can refer us to PhD in cognitive performance and neuron science if needed
• Look into cognitive brain science
  • What affect focus and attention  

phillip.thomas2@socom.mil

• Want the product to answer the following questions:
  1. How many people are going through pipeline in a day?
  2. How satisfied are trainees interested?
  3. What are the machines that are used the most?

• ROI, bottom line – (they are investing money and want to know how money is creating results)

oscar.gonzalez@usuhs.edu

• time series analysis, want to see changes over time
• comparisons over different groups (age bracket, ranks, years of experience, MOS/job description)

• Want the product to answer the following questions:
  1. What is the exercise that provides the biggest performance increase?
  2. What is the performance machine/training/process that creates the biggest change?
  3. Who are the people that have the biggest change and smallest change?
  4. When do people start showing performance training over time?

constance.garcia.ctr@socom.mil

• The ability to run statistical analysis
• Really like how the prototype shows who has “touched the data”
• Use excel to run basic analysis with a set of metrics for most of the time
• SPSS isn’t on the network so they must use the software on a stand alone computer

mike.s.williams1@gmail.com

• 75 Ranger Regiment: as new soldiers came in, they would put them where they were relative to the baseline

-”the average soldier can do 65 pushups per minute” -> here’s a regime on how you can get to the average pushup

• we need to create a normal distribution curve for people who complete the training
• Reach back out to Major Mike Williams when we need help w/ statistical analysis

gdale@duke.edu

• Three focus areas for Duke sports: leadership, culture, performance
• Works with students on: expectations, evaluation, awareness of consequences
• Teach students how to be aware of their nervous feelings and physical changes to perform better

Daniel.gajewski@socom.mil

• Issues still to tackle: data centralization from multiple sources, time stamping, aggregation…
• Finding a way to get data from the field back to the lab is ideal. Hardware solution is still necessary.
• Synchronizing data to an event during training is more important than having the program run advanced statistical analysis. (for Dan at least)

trevor.obrien@socom.mil

• Would like to see graphical user interface, how it was developed. Step by step. (less useful right now since it’s more of a visual representation)
• Showing what different profiles for different users would look like would be useful.
• Making sure the user interface is user friendly is key.
• Trevor is more of a hands on learner, could help us with development if we let him in.
• Provided some data on spear that may be helpful.
• Unrelated: Spear is likely going to get nixed, but could be helpful for us to use.

Frederick.d.dietrich@socom.mil

• Really likes how simply and intuitive the software is to use.
• Nexus 10 has a biotrace+ software that connects all sorts of devices:
  • EEG
  • ECG
  • Galvanic Skin Conductor
  • EMG
  • Blood Oxygen Level
  • Blood Volume Pulse
  • Extremity Temperature
  • SCP (Brian Score)
• Kubios is the software that gathers information from the Nexus
• Look at MindMedia

Frederick.d.dietrich@socom.mil

• Concerned about the data storage capabilities of the platform – they had to move away from excel b/c it couldn’t store the data
• Merging, regression, multiple analysis of variance is important
• Emphasized that auto population of data is pivotal to her job

james.arp@socom.mil

• Concerned about data storage and scalability

vatrina.mardre@socom.mil

• We need to figure out if our device will be on the lang or NIPR networks.
• Requirements for approval: cannot create vulnerabilities, must be compatible with Windows 10, cannot create risk
• Bluetooth can be approved, although it is hard.

majarth@gmail.com

• Military regulations are unclassified and open to the public -> ask LTC if he can send us rule regulation governing connectivity standards
• Interview someone who pushed for new connectivity rules
• Military bureaucracy creates more stringent rules as there are more and more rules as you go down the chain of command

rachfef@hotmail.com

• Advisory Board – works specifically in healthcare consulting, look at what information or research on what healthcare is doing
• Talk to people at Duke Hospital – rehabilitation department
• Soldiers at Walter Reed National Military  Medical Center are meeting with so many doctors, and they have a central database for which to track each patient.  

greg@duke.edu

• Different radio frequencies mean that we need to time sync
• Proprietary information from different sensor makers will make synching difficult.
• Arduino syncs by sending out an orientation pulse from each device and then it gets a timestamp
• Tobii (lots of data), hrv (very little).

yiyao.yuan@duke.edu

• Recommended Firefly hardware DIY platform
• Recommended tutorial for learning circuit design
• Willing to help with hardware prototyping if we needed it

mheath17@gmail.com

• Conduct more MVPs and really try to understand your problem sponsors prototyping problem

kyle.janson@duke.edu

• Willing to put us in contact with other ECE people working on data problems
• Might not need to use a microcontroller. Consider looking at 3rd party companies that offer data integration but not necessarily collection

mark.palmeri@duke.edu

• Learning the skills to create this component/ device might take more time than is allotted to you. It depends on how much time the team has willing to give.
• Willing to give time to go over possible device companies that are doing data integration

• Hardware solution (chip set) could work, but may only be a short term patch with high future sustainment costs.
• A network/web app solution, ideally with a 3rd party vendor would be ideal.
• A thin client solution, where the software is hosted on a server and the devices only need storage and RAM could save up to $15 million.

• SEAL school has similar issue with attrition, A/S at 87%.
• We should know whether or not attrition is coming from people giving up or from failing out. Special operator recruitment has changed in part due to pop culture influence, this leads to recruits who want to be hot shots and get disillusioned early on (in that case they are getting the wrong people).
• Focus could change who is being recruited so that they get the right mental profile. Even though this seems like an obvious problem but school is generally blind to it (outsider perspective useful).

1. Compatibility of raspberry Pi with the current three hardware
2. Use case scenario of the product (environmental factor)
3. Philp would be extremely helpful for us if we actually want to deliver a working product (Even by the end of day, all we deliver is just a MVP, it would be great we make detail recommendation and plans to LTC Thomas and his team about how to carry this project further)

1. Specific job – develop and acquire capabilities within Special Operations command
2. Might be good person to speak to once we have a MVP

1. Need a system that could incorporate both the biometrics data and the cognitive data
2. Other unit is also using tablet for self-assessment, HDP is working on funding, the fastest turnaround would be 6 month
3. Would like us to research and provide a comparison on different tablet offering, ex: iPad vs Samsung
4. Military like action item, end the report/presentation with a list of action items
5. Present the solution in an incremental plan

• Step 1:
• Step 2:
• Step 3: if have $$$/enough resources, you could..

1. 92% of trainees get to +1 area after 3 week training program
2. Human feedback is the best. It can’t be beat by a computer. Trainees need to interact with people from that culture.
3. Government employees here not contractors.
4. Military personnel are constantly on rotational assignment (3 years)

– on a scale of 1 – 10 of how helpful would this be? 10

-Alexandra and Constance will be using the SD cards from the Zigbee & do analysis from time to time rather than constant

1. Identify other commercial solution
2. Asking for spec
3. Bottom line: our proposed solution need to be flexible enough to accommodate all kinds of data, including cognitive data and allowing data team to manually input self-assessment survey
4. LTC Thomas wonders where the repository of the data that going to end up

– cpc’s and other contractors working with the navy SEALs were underpaid/undertrained, led to friction.

– lack of communication between SEAL school and SWIC, even if they were working on the same thing.

-nail in the coffin for the SEALs  was privacy concerns, killed any data sharing.

– He has connections to other people who did work the SEALs. (Potential leads).

Workforce Sciences Institute

– worked with the Navy and used ILM to help identify key skills to improve upon.

– didn’t work with special forces, but has contacts that may be useful.

1. The facility will accommodate our proposed solution
2. Getting rid of the middle step of placing micro-computer to soldiers’ jacket
3. Talking to industry expert and learning how tech company solve similar problems

1. A device that allows quick insights would be helpful for the soldiers.

Class size of 50-60, 50% success rate.

Only 33% of enlistees and 40% of Officers make it through assessment and selection.

Highest attrition from physical/mental assessment and culminating exercise.

Is the data we are collecting useful for providing an MVP?

Cadre operate fairly idiosyncratically. Since they don’t all measure the same things this leads to confusion. Soldiers are often told they are being measured one way when it is actually something else is being measured.

Recruit Class was 1200 this year.

PIT saved $1 million this year.

determine what format the solution should be based on.

Focus on really understanding quantitative metrics for green beret training

What insights do we want the instructors to be able to easily obtain from the data?

Focus on wearable tech.

There is an entire unit working on the bidding process, (highly complex).

Problems with shrinking recruiting class, both witin the army for SWIC and in the general US population.

Any solution is cost prohibitive, not only in procurement but also in time spent on implementing and maintaining any data system.

Same biometrics data may means different things as each individual is unique. How to assess data can be much more difficult than how to collect them  

Brass want more graduates, cadre want better quality graduates.

New generation of soldiers are different to train, causes friction.

USSOCOM, USASOC, Army

We have to justify the value of the training programs to the 2,3, and 4 star general commands

cooperation/information sharing is generally facilitated by relationships, if they aren’t there, then generally different units won’t communicate.

Technology is changing the landscape of the federal gov and DoD

Focus on the cultural aspect of getting military

SWEG doesn’t have a committee that makes data-driven decisions.

Peer feedback, instructor feedback, and some data informs human performance results.

30 day ideal innovation turnover for new devices. He would like to be able to test new hardware/software without jumping through the regulation/ permission hoops.

Use case:

Immediate feedback mechanism as the top use case priority from Commander Rice point of view. Have expert as a reference point for novices

using big data/database to make key decision such as selection process/recruiting. Track each trainee’s performance/data throughout the training process

Brass want more graduates, cadre want better quality graduates.

New generation of soldiers are different to train, causes friction.

Strength and conditioning coaches have to download to share the database with other data users (physical therapists, CPC’s, data analysts, research psychologists)

Database’s real time feedback has been helpful in readjusting physical training

No protocol for how physical therapists make data-informed decisions

Did not interact much with CPC’s, data analysts, and research psychologist

Acquisition for different accounts at SWEG is siloed which means that different solutions will not arrive at the same time.

The current output of that eye tracking

device is just a video

Has another software platform for further analysis but they haven’t start yet

US Army Special Operations Command

“CONSTANTLY. I CAN’T EXPLAIN TO YOU HOW OFTEN IT HAPPENS”

They tend to type all the data all over again. This is a constant problem. Army are multi-service.

Working on concerns from data analysts about the large amount of time spent toward data input

• Testing process before getting approval
• 3-6 month on average to get approval
• The new equipment need be on the Disa, if not need to submit to Disa (take 2-3 month), then go through certificate to operate

-CPC -> data analyst ->research psychologist is ideal rather than what is happening now

-CPC: 70% is working with data (cleaning/adjusting spreadsheet/running analysis)

20% is actual data collection of it, do hard copy,

10-15% of the time spend with other CPC, learning  what are they working on

5% Research, finding the norm, what other methods

-only 40-50% CPC’s collect performance enhancement data

-CPC’s drop off data in person, and data analysts have to manually put in data (data input takes up 70% of her time inputting data)

-sometimes CPC want data to be inputted and not analyzed

– Besides lacking an integrated data platform(from the tech side), it seems human factors(lack of communication and collaboration) and inefficient processing (manual input, no standard protocol of the format of the data) also contribute to the problem.

1. Curtis sees the organism approach as a more efficient manner of human development
2. Problem is in resource prioritization. HDP has different priorities. Training, injuries. His priority is within longer term, longitudinal studies.
3. DATA MANAGERS DEPARTMENT IS NOT SUPER COMPLEX. LOTS OF MANUAL INPUT.
4. Good contact for people outside of HDP

tommy.duncan@ptp-llc.com

-Oscar has been sitting in on meetings about religiousness for the performance integrator (moving toward the ideal model now)

-discrepancies with how many CPC’s collect data – this person estimates 50% collects data

-Security is the primary concern, more so than sharing. Data being abused is already a problem. Countermeasures from cyber attacks and EMPs are also a consideration.

-Any solution we provide should be evaluated on whether or not more Spec Ops soldiers are coming out of the program. Ineffective data collection is actually hurting some recruits, washing them out based on technicalities

-CPC’s are well integrated with each other but not with the data analysts and research psychologists

-CPC send all the raw data to the data analyst -> no standard procedure for CPC’s of what to do with data  

• Special workfare school split between special training, education and medical
• Two option: a research report either go to SWIC HQ or publish to a journal
• Clinical psychologist(similar to industrial psychologist) collecting psychological data at an earlier stage, does not collaborate well with CPC.

-Accessing the server is hard – “not a day goes by that I don’t have trouble with the system”

-Talk to five people: sports medicine, physical therapist, strength and conditioning, dieticians, CPC, interpersonal coaches, spiritual advisors

(SWCS)

1. New facility is still in development and design
2. Hard to get approval to share information outside of the organization(intranet to internet)

-SPEAR: both a hardware and software solution

Special Warfare Education Group

(SWEG)

-SPEAR: not user friendly enough

Special Warfare Education Group

(SWEG)

-Come back to him after a few interviews.