Java Performance:Biggest Mistakes


The Presentation inside:

Slide 0

More on http://blog.dynatrace.com Dynatrace Free Trial: http://bit.ly/dttrial Andreas Grabner - @grabnerandi Java Performance: Biggest Mistakes


Slide 1

And this is where I am from


Slide 2


Slide 3


Slide 4

Can you tell my age by looking at my first computer?


Slide 5

That’s why I ended up talking about performance


Slide 6

Nobody wants this …


Slide 7


Slide 8

Nor this …


Slide 9


Slide 10

As it leads to this …


Slide 11

The “War Room” Facebook – December 2012


Slide 12

And potentially to this …


Slide 13


Slide 14

And this …


Slide 15


Slide 16

And that’s why Business doesn’t like it either …


Slide 17

~80% of problems caused by ~20% patterns YES we know this 80% Dev Time in Bug Fixing $60B Defect Costs BUT


Slide 18

6 Situations on WHY this happened, HOW to avoid it


Slide 19


Slide 20

“Blindly” (Re)use Existing Components


Slide 21

Requirement: We need a report


Slide 22

Using Hibernate results in 4k+ SQL Statements to display 3 items! Hibernate Executes 4k+ Statements Individual Execution VERY FAST But Total SUM takes 6s


Slide 23


Slide 24

Not every Architect makes good decisions


Slide 25

Symptoms HTML takes between 60 and 120s to render High GC Time Assumptions Bad GC Tuning Probably bad Database Performance as rendering was simple Project: Online Room Reservation System


Slide 26

Developers built own monitoring void roomreservationReport(int roomid) { long startTime = System.currentTimeMillis(); Object data = loadDataForRoom(roomid); long dataLoadTime = System.currentTimeMillis() - startTime; generateReport(data, roomid); } Result: Avg. Data Load Time: 45s! DB Tool says: Avg. SQL Query: <1ms!


Slide 27

#1: Loading too much data 24889! Calls to the Database API! High CPU and High Memory Usage to keep all data in Memory


Slide 28

#2: On individual connections 12444! individual connections Classical N+1 Query Problem Individual SQL really <1ms


Slide 29

#3: Putting all data in temp Hashtable Lots of time spent in Hashtable.get Called from their Entity Objects


Slide 30

Custom Measuring Was impacted by Garbage Collection Just measured overall time but not # SQL Executions Learn SQL and don’t use Hashtables as Workaround Lesson Learned void roomreservationReport(int roomid) { long startTime = System.currentTimeMillis(); Object data = loadDataForRoom(roomid); long dataLoadTime = System.currentTimeMillis() - startTime; generateReport(data, roomid); }


Slide 31


Slide 32

Implementation Flaws


Slide 33

Business Impact requires Action!


Slide 34

Solution: Cache to the RESCUE!!


Slide 35

Implementation and Rollout Implemented InMemory Cache Worked well in Load Testing


Slide 36

Result: Out of Memory Crashes!! Still crashes Problem fixed! Fixed Version Deployed


Slide 37


Slide 38

Disconnected Teams


Slide 39

“Teamwork” between Dev and Ops SEV1 Problem in Production Need access to log files Where are they? Can’t get them Need to increase log level Can’t do! Can’t change config files in prod!


Slide 40

Solution: Implement a Custom “On Demand” Remote Logger


Slide 41

Implementation and Rollout Implemented Custom Logger Worked well in Load Testing


Slide 42

What happened? ~ 1Mio Lock Exceptions in 30 mins


Slide 43

Root Cause: A special WebSphere Setting!


Slide 44


Slide 45

“Deployment” Gone Bad!


Slide 46

Test Environment Production Environment 8x slower 3x more SQL


Slide 47

Test Environment Production Environment Hibernate, Classloading, XML – The Key Hotspots Hibernate, Classloading, XML – The Key Hotspots I/O for Web Requests doesn’t even show up! That’s Normal: Having I/O for Web Request as main contributor


Slide 48

Top Contributor Class.getInterfaces Called from Hibernates FieldInterceptionHelper These calls all originate form thousands of calls to find item by code


Slide 49

Top Methods related to XML Processing Classloading is triggered through CustomMonnkey and the Xalan Parser Classloading is triggered through CustomMonkey and the Xalan Parser


Slide 50


Slide 51

#Push without a Plan


Slide 52

Mobile Landing Page of Super Bowl Ad 434 Resources in total on that page: 230 JPEGs, 75 PNGs, 50 GIFs, … Total size of ~ 20MB


Slide 53

Fifa.com during Worldcup http://apmblog.compuware.com/2014/05/21/is-the-fifa-world-cup-website-ready-for-the-tournament/


Slide 54

Waiting for Bug Reports


Slide 55

Look behind the scenes


Slide 56


Slide 57

Performance Clinic: Bring Your Own App When: Thursday, December 18th Lunch Session: NOON – 3PM After Work Session: 4PM – 7PM Where: Dynatrace Office 404 Wyman Street, Waltham Register: http://bit.ly/onlineperfclinic


Slide 58

Quick Demo


Slide 59

Get it here: http://bit.ly/dttrial YouTube Tutorials: http://bit.ly/dttutorials Live Q&A Sessions: http://bit.ly/onlineperfclinic Contact me: [email protected] - @grabnerandi Special Offer: SaaS VIP Program: http://bit.ly/dtaasvip 30 Days Dynatrace Free Trial


Slide 60

Andreas Grabner Dynatrace Developer Advocate @grabnerandi http://blog.dynatrace.com


×

HTML:





Ссылка: