Rise of the Machines - Automate Your Development


The Presentation inside:

Slide 0

Rise of the machines Automate your development @svenpet


Slide 1

Disclaimer Scripts Chef, Puppet, etc better build pipelines


Slide 2

Disclaimer Ideas through examples


Slide 3

Rise of the machines


Slide 4

ROBOTS are coming to our day to day lives


Slide 5


Slide 6


Slide 7


Slide 8


Slide 9

Rise of the machines IN Software Development


Slide 10

Sven Peters Atlassian Evangelist svenpet.com @svenpet


Slide 11

Automation in Software Development my personal story GUI builder 1996 1998 2001 build scripts (make) JUnit Eclipse (code generation) 2004 2006 2005 Hudson ant -> later maven 2009 Puppet 2013 Docker


Slide 12

What are we automating right now? Deployments Build & Tests Small scripts


Slide 13

What are we automating right now? Support Code Reviews Merging Stand Ups Ops Engineering Health


Slide 14

Bots everywhere Coding bots Test bots Ops bots Service bots Report bots Doc bots


Slide 15

I’m new Welcome… or


Slide 16

Happy installing & setup… for the next 8 hours Add ons JDK IntelliJ Git Postgres Scala


Slide 17


Slide 18


Slide 19


Slide 20


Slide 21


Slide 22


Slide 23


Slide 24

Meet Agent Charlie


Slide 25

Meet Download Install Configure Agent Charlie


Slide 26

Meet Download Install Configure Agent Charlie


Slide 27

Puppet, Shell & Ruby Works with OSX & Ubuntu IntelliJ IDEA Atlassian Plugin SDK JDK Bitbucket SSH Keys Maven Git PostgreSQL Mercurial Stash Keys HipChat and much more… HomeBrew


Slide 28

Without Agent Charlie With Agent Charlie


Slide 29

Coding bots Automate your Coding phase


Slide 30

27’’ is not enough


Slide 31

Repo manager IDE Chat Email Issue Tracker Command line


Slide 32

Issue tracker workflow TODO IN PROGRESS IN REVIEW DONE


Slide 33

Developer workflow BRANCH CODE REFACTOR PULL MERGE


Slide 34

What you should do TODO BRANCH IN PROGRESS CODE IN REVIEW REFACTOR PULL DONE MERGE


Slide 35

What you do TODO BRANCH IN PROGRESS CODE IN REVIEW REFACTOR PULL DONE MERGE


Slide 36

Keep the team updated!


Slide 37


Slide 38

Smart commits mention issue_key and #trigger the issue tracker Github JIRA YouTrack Trello Bitbucket


Slide 39

Control the issue by commits TODO IN PROGRESS IN REVIEW DONE JRA-566 #start-Progress JRA-566 #start-review fixes #566


Slide 40

No tool switching Robot is just receiving commands


Slide 41

Controlled by action TODO IN PROGRESS IN REVIEW DONE branch commit pull request create review merge close review


Slide 42


Slide 43

Our workflow TODO IN PROGRESS branch IN REVIEW pull DONE merge


Slide 44

branch master JRA-554-bugfix pull merge


Slide 45

branch pull commits code directly to the master branch! master merge


Slide 46

branch master pull merge


Slide 47

branch master JRA-554-bugfix pull merge


Slide 48

branch master JRA-554-bugfix pull merge


Slide 49

bad surprise sh*t happens, but can be avoided


Slide 50

no bad surprise sh*t doesn’t happen that often


Slide 51

branch pull merge master JRA-554-bugfix don’t branch from a red build


Slide 52

branch pull merge master JRA-554-bugfix invite the best reviewers


Slide 53


Slide 54

Recent comitters or author


Slide 55

auto suggest reviewer


Slide 56

branch pull master JRA-554-bugfix minimum 2 approvals merge


Slide 57

branch pull merge master JRA-554-bugfix unapprove reviewers when code changes


Slide 58

The need to update older versions


Slide 59

Support older versions master Version 1.1 Version 1.0


Slide 60

Support older versions master Version 1.1 Version 1.0 JRA-234-hotfix


Slide 61

Support older versions master Version 1.1 Version 1.0 JRA-234-bugfix boring!


Slide 62

Huhu I can do it!


Slide 63

auto merges master Version 1.1 Version 1.0 JRA-234-bugfix


Slide 64

Test bots SMARTER Test Automation


Slide 65

&:hover { text-decoration: none; color: @invertedBodyTextColor background-color: #366ca6; cursor: pointer; }


Slide 66

&:hover { text-decoration: none; color: @invertedBodyTextColor background-color: #366ca6; cursor: pointer; } Static code analysis can catch this! Findbugs - Checkstyle - PMD


Slide 67

Listen to what Freud tells you about your source code


Slide 68

Freud Bot &:hover { text-decoration: none; color: @invertedBodyTextColor background-color: #366ca6; You should use a variable instead for #366CA6 . I suggest that you use a variable of similar color: @globalHeaderPrimaryActionFallbackColor @globalHeaderTopColor cursor: pointer; } @primaryLinkColor


Slide 69

Freud Bot for Pull Requests


Slide 70

Freud Bot less review work makes suggestions


Slide 71

We are trying hard to write awesome code


Slide 72

package com.miguelcatalan.materialsearchview; MyAwesomeClass import android.app.Activity; import android.widget.ListView; import android.widget.RelativeLayout; import android.widget.TextView; import java.lang.reflect.Field; import java.util.List; /** * @author Miguel Catalan Bañuls */ public class MaterialSearchView extends FrameLayout implements Filter.FilterListener { private MenuItem mMenuItem; LOC: 32 private boolean mIsSearchOpen = false; public MaterialSearchView(Context context) { this(context, null); } public MaterialSearchView(Context context, AttributeSet attrs) { this(context, attrs, 0); @Override public void onTextChanged(CharSequence s, int start, int before, int count) { mUserQuery = s; startFilter(s); MaterialSearchView.this.onTextChanged(s); } }


Slide 73

One month later


Slide 74

package com.miguelcatalan.materialsearchview; import android.app.Activity; import android.content.Context; MyAwesomeClass import android.content.Intent; import android.content.pm.PackageManager; import android.content.pm.ResolveInfo; import android.content.res.TypedArray; import android.graphics.Rect; import android.graphics.drawable.Drawable; import android.os.Build; import android.os.Parcel; import android.os.Parcelable; import android.speech.RecognizerIntent; import android.text.Editable; import android.text.TextUtils; import android.text.TextWatcher; import android.util.AttributeSet; LOC: counting…. import android.util.Log; import android.view.KeyEvent; import android.view.LayoutInflater; import android.view.MenuItem; import android.view.View; import android.view.inputmethod.InputMethodManager; import android.widget.AdapterView; import android.widget.EditText; import android.widget.Filter; import android.widget.Filterable; import android.widget.FrameLayout; …one month later import android.widget.ImageButton; import android.widget.ListAdapter; import android.widget.ListView; import android.widget.RelativeLayout; import android.widget.TextView; import com.miguelcatalan.materialsearchview.utils.AnimationUtil;


Slide 75

mSearchSrcTextView.setText(null); mSearchSrcTextView.requestFocus(); MyAwesomeClass if (animate) { AnimationUtil.fadeInView(mSearchLayout, AnimationUtil.ANIMATION_DURATION_MEDIUM, new AnimationUtil.AnimationListener() { @Override public boolean onAnimationStart(View view) { return false; } @Override public boolean onAnimationEnd(View view) { if (mSearchViewListener != null) { mSearchViewListener.onSearchViewShown(); } return false; } LOC: 986 @Override public boolean onAnimationCancel(View view) { return false; } }); } else { mSearchLayout.setVisibility(VISIBLE); if (mSearchViewListener != null) { mSearchViewListener.onSearchViewShown(); …one month later } } mIsSearchOpen = true; } /** * Close search view. */ public void closeSearch() {


Slide 76

Code over time 86% test coverage 20 deprecated methods 1045 old unit tests (JUnit 3) 27 compiler warnings


Slide 77

Code over time 83% test coverage 23 deprecated methods 1136 old unit tests (JUnit 3) 36 compiler warnings


Slide 78

Code over time 75% test coverage 32 deprecated methods 1289 old unit tests (JUnit 3) 45 compiler warnings


Slide 79

Developers don’t write bad code on purpose Developers are trying to solve problems


Slide 80

Calling Dr. Code measure constantly the engineering health


Slide 81

Calling Dr. Code measure constantly the engineering health


Slide 82

JUnit 3 JUnit 4 No more JUnit 3 tests! jira.stats.tests.junit.3.count


Slide 83

Inspect all the things jira.stats.tests.junit.3.count jira.stats.bundled.jars.in.plugins


Slide 84

Caught by Dr Code? API compatibility :( New JUnit3 test JS not minified You’ll enter the Hall of Shame!


Slide 85

You’ll enter the Hall of Shame! Wallboard - so everyone sees it!


Slide 86

Fixed a matrix? You’ll enter the Hall of Fame!


Slide 87

Old version New version public abstract class Task { public abstract class Task { public final void executeWith(Callback callback) { public final void executeWith(Callback callback) { execute(); execute(); if (callback != null) { if (callback != null) { callback.call(); callback.call(20); } } } } public abstract void execute(); public abstract void execute(); } }


Slide 88

Old version New version public abstract class Task { public abstract class Task { public final void executeWith(Callback callback) { public final void executeWith(Callback callback) { execute(); execute(); if (callback != null) { if (callback != null) { callback.call(); 20 callback.call( } ); } } public abstract void execute(); } } public abstract void execute(); }


Slide 89

Old version New version


Slide 90

Old version Hard to spot? New version


Slide 91

Image Compare Bot Compare images Alert on differences Easy reporting (on build) (build warning) (build report)


Slide 92

Our functional tests ran for h urs so we ran them parallel


Slide 93

waiting for the longest test block Test block 1 Test block 2 Test block 3 Test block 4 Test block 5 statically sized test blocks


Slide 94

Hallelujah Functional Test Balancer


Slide 95

Hallelujah T1 T6 T2 T8 T3 T4 T11 T13 T9 T7 T5 T14 T10 T 12


Slide 96

build status


Slide 97

build status


Slide 98

build status


Slide 99

Flaky Test Detector


Slide 100

Flaky Test Detector rerun test failing test successful test continue build test in quarantaine create issue


Slide 101

1888 flaky tests detected 817 quarantined tests fixed


Slide 102

no manual reruns no more ‘this one again’ moments Flaky Test Detector forces fixing flaky tests no log file searching


Slide 103

OPS bots Automated Operations


Slide 104


Slide 105

Ops problem now


Slide 106

Slow feedback loop prob ops dev


Slide 107

Faster feedback loop prob dev


Slide 108

Faster feedback loop knows which team -> service responsibility


Slide 109

Faster feedback loop But that’s a lot of configuration!


Slide 110

Faster feedback loop add team in deployment descriptor add connection to pager duty


Slide 111

Own your code end-to-end


Slide 112

Prepared for the release? WAIT… Release Notes?


Slide 113

Do you know which features really ship? Bugfix New Feature Improvement Release Branch


Slide 114

Release Notes Bot Merged JRA-345 Commit messages + = Issue information Release notes


Slide 115


Slide 116

Label in GitHub Issues Label in GitHub Issues


Slide 117


Slide 118


Slide 119

New features Improvements Fixed bugs


Slide 120

No surprises Human readable issue descriptions Think about customer value Issue Description as Release Notes


Slide 121

Service bots Automate your Service


Slide 122

Support ask for log file Developer scanning log files system knowledge


Slide 123

pause work


Slide 124

Support ask for log file Developer scanning log files system knowledge


Slide 125

Support ask for log file Hercules scanning log files system knowledge


Slide 126

Teaching the robot


Slide 127

Hercules get’s smarter suggest read update


Slide 128

{ Well f*%k me THE ROBOT was right! }


Slide 129

Your boss has a really important task for your team


Slide 130

Everyone is working from home today


Slide 131


Slide 132

by


Slide 133


Slide 134

With chat you can reach everyone immediately


Slide 135

How do I download the Sprint report? How do I set up a build for my feature branch? All these questions! What Java versions do we support?


Slide 136

!ask Self service for chat I’ll look it up for you


Slide 137

!ask the knowledge base first!


Slide 138

REPORT bots Manual Reporting sucks


Slide 139

Automating Stand Up s ?


Slide 140

too long setup VC calls Stand Up timezones Problems remind people


Slide 141


Slide 142

Music - Start at 09:29:00 Crontab! Never miss a stand up


Slide 143

Video - Open at 09:30 No video set up time


Slide 144

Stand up report bots distributed colocated different timezones


Slide 145

Status update


Slide 146

Team update


Slide 147

Stand up bot


Slide 148

New issue Exception Chat Bots everywhere Build failure Deployment


Slide 149

ΩΩΩΩ Builds


Slide 150

ΩΩΩΩ Deployments


Slide 151

ΩΩΩΩ Exceptions


Slide 152

Chat Bots everywhere Fast feedback loops Team knowledge Start discussions


Slide 153

The build failed New deployment


Slide 154

The build failed New deployment


Slide 155

The build failed New deployment


Slide 156

New important issue New deployment


Slide 157

New important issue New deployment


Slide 158

Get the big picture New important issue New deployment


Slide 159

Get the big picture


Slide 160

e ck u tb i o b .b rd a h as tl /a :/ tp t g/ or t. Every team has one or more


Slide 161

e ck u tb i g/ or t. .b rd a o b as tl /a :/ tp t Every team has one or more h see what’s going on point directly


Slide 162

All kind of boards Advocates Developer Support


Slide 163

Bots everywhere Coding bots Test bots Ops bots Service bots Report bots Doc bots


Slide 164

Writing automation code Reinvent the wheel? Very specific problem Lots of unmaintained projects


Slide 165

No. 1 Problem


Slide 166

Technical debt 10% work time for better engineering health


Slide 167

Technical debt 10% work time for better engineering health


Slide 168


Slide 169

Motivation Success Your Problems Fast feedback Automation is serious fun! Languages Technologies Go - Ruby - PHP - Python Puppet - Chef - Ansible - Docker


Slide 170

From scripts on a developer hard disc


Slide 171

to professional automation


Slide 172

Add code to Source Control


Slide 173

Do Code Reviews for every change


Slide 174

Write Tests


Slide 175

Have Deployment Scripts


Slide 176

Keep your automated systems under control


Slide 177

Keep your automated systems under control Lots of different places


Slide 178

Instances for automation services


Slide 179

We use labels for better overview st co service ow ne r e tr n ce


Slide 180

Micros <our own PaaS> Python Docker Java Go


Slide 181

Micros Simple to create & register micros service:create <service-name> Owner Service descriptor PagerDuty Cost Centre


Slide 182

Were are we heading?


Slide 183

Were are we heading? better conclusions collecting more data Better customer insides


Slide 184

Were are we heading? less time updating & searching integrated tools Faster dev feedback loops


Slide 185

Were are we heading? deeper code analysis smarter tests Better Quality


Slide 186

Building software Creating customer value is a is an craft Parts can be automated art Hard to automate


Slide 187

Rise of the machines …Skynet became self aware


Slide 188


Slide 189

@svenpet svenpet.com/talks find slides here I’ll be back…


Slide 190

“VW Golf TDI Clean Diesel was 2010 8983" by Mariordo Mario Roberto Duran Ortiz - Own work. Licensed under CC BY-SA 3.0 via Commons - https:// commons.wikimedia.org/wiki/ Photo Credits Backyard Series by syauqee mohamad "http:// www.flickr.com/photos/[email protected]/6945260735" Tesla Model S by Chrishmt0423 from flickr Burnt & Abandoned Computer "http://www.flickr.com/ photos/[email protected]/13262802714"


Slide 191


×

HTML:





Ссылка: