Performance Culture

 What is a Culture:

"culture" is a versatile term that can refer to the shared characteristics and behaviors of a group of people, whether at the societal, organizational, biological, or artistic level.

What is a Performance Culture:

• Share the same perf vision/goal
• Share the same perf language (basic concepts/terms/rules)
• Share the same perf process
• Share the same perf methodology
• Share the same perf toolkit
• Team work
• Continuous measure, monitoring and improve, never satisfied
• Team start looking at data,/numbers, questioning and discussing performance stuff

Response Time Distributions Vs. Performance Metrics

•  Right-Skewed Distribution

P90: 4s

P50: 1s 

AVG: 1.88s

• Normal Distribution

P90: 4s

P50: 3s 

AVG: 3s

•  Left-Skewed Distribution

P90: 5s

P50: 4s 

AVG: 4.1s

• Bimodal Distribution

P90: 8s

P50: 3s 

AVG: 3.9s

Random Quotes

• "Everything which is measurable needs to be measured, properly!"

• “In terms of performance issue, it appears that the application is running slowly, but more often it indicates something might implement wrong!”

• "There is no Mystery or Magic for the slowness, must be (a) reason(s) you see or you may not see (yet)!"

• "Isolation, isolation, isolation!"

• "We are not concerned with all samples as it can be overwhelming and sometimes misleading. Our primary focus is solely on improving TP90 for each individual product or app."

• "Care about the Facts! The world is a certain way, and now using facts as the baseline, we can try to figure out how to make things better."

• "Keep the momentum and complete the loop"

• "Do not mix everything together! Tuning one thing at a time, and measure it separately. "

• "Build Visualization is to help me understand the systems and share with others"

• "If you can not decide how many samples you gonna look at for your analysis, choose 5,  and ignore the min/max ones, it will give you 93.75% to locate representative range of all samples"

• "Fixing high-traffic but poor-performing areas(so called high cost) of your site will help lift your overall metrics, since the total resources are scarce and sometimes they are competing with each other"

• "Any performance or load tests gives you the confidence rather than concrete numbers, even if you already model 'perfectly' and conduct it carefully"

• "Bigger Hardware is not always faster, but it can usually handle more load"

• "The more data you accumulate, the more obvious the facts will become" 

• "Performance numbers under high Error% is meaningless, fix the error first!"

• "If you run the ETL job, you monitor the site! lol"

• "The Capacity issue always leads to the slowness, however, the slowness is not always caused by the Capacity. So they are relatives but not the same topic. But try to fix (all) performance issues first before you evaluate your capacity!"

• "API design should always keep applications in mind with a limited boundary, otherwise it is useless or abused"

• "If you do not understand it, do not use it!" – My 10 year old daughter told me...

• "Don't blame yourself, it's not entirely your fault, it's just that the reality is different from what you thought or expect.."

• "The correct measurement is not only able to find the issue, but also prevent the issue from persisting"

• "There is no solution, there are only trade-offs!" – Thomas Sowell 

• "The incomprehensible should cause suspicion rather than admiration." – N. Wirth

• "Go where the symptom is , and look at it... At the ACTUAL symptom" – Cary Millsap

• "It is not enough just to make tools available to developers; those developers also have to have THE WILL and THE SKILL to use them" – Cary Millsap

• "It's not data or intuition, it's data and intuition" – Ivy Ross
• "Site Speed isn't something you fix once and walk away from" – SpeedCurve
• "We need simplifications. They include methodologies and effects models, both implicit and explicit." -- Gerald M. Weinberg, And we need understand principles too
• "Increasing variability makes the utilization and latency graph get worse faster"
• The Data-Information-Knowledge-Wisdom (DIKW) pyramid, also known as the DIKW hierarchy, is a model that represents the relationship between data, information, knowledge, and wisdom -- wikipdia "DIKW pyramid"
• "The better the job you have done in finding a path for yourself, the more boring and predictable your life is going to be" -- Jerry Seinfeld
• "Unpredictable is one of the most challenging things in Performance Engineering!"
• "The great performance number looks flat and smooth, the bad performance number looks fluctuation. The slow ones may looks flat and smooth too, I also call it great since they are usually easier to tune faster!"
• “Take performance seriously, but there is a good way to do performance improvement, and that needs a disciplined measured process!” – Martin Fowler
• "If you do not care about performance, the slowness will knock at your door someday, sooner or later"