DevOps Best Practices - a call to action
CIO Working Group formed to explore DevOps on Campus
Some folks believe they already do DevOps.
State of DevOps at Stanford
Desire for a standard definition of DevOps across campus.
We looked at the Wikipedia definition.
Benefits of DevOps - why do this thing?
Biz & Academic Continuity
Infrastructure As Code; Repeatability and Predictability.
Accountability & Supportability; shared responsibility
Security...bake it in to the product lifecycle
Agility; faster deliverables, more innovation.
Hans sez these are important, but apparently lacking from the Wikipedia definition.
Many challenges identified.
- Cultural is huge. Dev vs. Ops., not wanting to learn new things.
- transparency is hard.
- fear of loss (of control)
- Talent retention; myth of Full Stack Developer. It's a full stack TEAM.
- Cost of change; need up-front investment. Tools, personnel, processes.
- Pressure to deliver on existing commitments limits time for change/innovation.
Some successes already identified.
- But we're all still learning
- And we're all doing it differently, often in the same dept.
Can we re-invigorate the DevOps COP?
What does that mean? How can we increase participation? Answer uncertain.
What's our final message to the CIO Council?
- Finding people with the broad range of skills is very difficult.
- Full Stack Dev vs. Full Stack TEAM.
- Do people *want* to learn new stuff? They should. But if they don't, that points to a personnel issue.
- Can't be successful as only grassroots - it needs mgmt support to flourish.
Groups may need to be restructured. Physical/geo space limitations, org issues.
Resistance to change.
Embedding Ops and Dev together on projects.
"Devs should pay for the wall."
DevOps takes a team.
How do we maintain cohesion across working groups on campus? To UIT?
We need slack in the schedule to be agile. Planning sprints a year out can be bad for handling emerging problems.
Agile...many different flavors. Kanban, Scrum, etc. Don't be prescriptive.
Time time time needed to implement tools - finance models need to take that into account. How much time should you take to automate a process? Consult xkcd.
We need people who are curious and don't just want to do the one thing (I'm a Java programmer! I don't want to know about anything else, ever!) They don't need to become all-stars, they just need to be curious and willing to learn enough about something new to know how to apply it.
Tools can be used to allow devs to deploy to prod safely. Unit tests, staging, etc. CI/CD. But not everything can use CI/CD automation.
QA needs to be developers. "Traditional" manual QA is not a thing.
Test driven development can help. But it needs mgmt to enforce it; left to their own devices, Devs might not write tests.
Challenge - How do we do UAT in Agile/DevOps?
Drive to Cloud is a great opportunity to re-invent our processes.
- Greenfield is an opportunity to do new stuff!
MUST BE ALLOWED TO FAIL
FAILURE IS HOW WE LEARN
FAIL FAST FAIL EARLY
CELEBRATE LESSONS LEARNED
There is a feeling that Stanford does NOT tolerate failure. Cultural issue.
We need to curate institutional, not indiviual, knowledge.
Automation reduces risks and increases velocity.