Felienne Delft University of Technology


The Presentation inside:

Slide 0

Felienne Delft University of Technology (@felienne) Putting the science in computer science This slidedeck is about the science in computer science c


Slide 1

Felienne Delft University of Technology (@felienne) Putting the science in computer science This slidedeck is about the science in computer science


Slide 2

We all love science, right?


Slide 3

We love ‘crazy’ science like this http://the-toast.net/2014/02/06/linguist-explains-grammar-doge-wow/


Slide 4

We love ‘crazy’ science like this And this http://allthingslinguistic.com/post/56280475132/i-can-has-thesis-a-linguistic-analysis-of-lolspeak


Slide 5

What is science?


Slide 6

What programming language is the best? So, with science, we should be able to answer questions about the universe, like this one.


Slide 7

You expect You’d expect computer scientists to somewhat respectfully debate this.


Slide 8

You get interpreted >> compiled JavaScript 4 ever! Pure is the only true path C++ is for real coders PHP sucks Pascal is very elegant You’d expect computer scientists to somewhat respectfully debate this. Unfortunately, reality is more like this.


Slide 9

State of the debate is not so scientific


Slide 10

Idea: ask people But, some people are trying! In this slidedeck I’ll highlight some of the interesting results those people have found so far.


Slide 11

Researchers at Berkeley have conducted a very exptensive survey on programming language factors. http://www.eecs.berkeley.edu/~lmeyerov/projects/socioplt/viz/index.html


Slide 12

13.000 responses Researchers at Berkeley have conducted a very extensive survey on programming language factors. They collected 13.000 (!) responses, and their entire dataset is explorable online http://www.eecs.berkeley.edu/~lmeyerov/projects/socioplt/viz/index.html


Slide 13

They founds loads of interesting facts, I really encourage you to have a look at their OOPSLA ‘13 paper (Empirical Analysis of Programming Language Adoption) My favorite is this graph, factors for choosing a particular language. http://www.eecs.berkeley.edu/~lmeyerov/projects/socioplt/viz/index.html


Slide 14

They founds loads of interesting facts, I really encourage you to have a look at their OOPSLA ‘13 paper (Empirical Analysis of Programming Language Adoption) My favorite is this graph, factors for choosing a particular language. Notice that the first factor that has something to do with the language is on the 6th place. http://www.eecs.berkeley.edu/~lmeyerov/projects/socioplt/viz/index.html


Slide 15

They founds loads of interesting facts, I really encourage you to have a look at their OOPSLA ‘13 paper (Empirical Analysis of Programming Language Adoption) My favorite is this graph, factors for choosing a particular language. Notice that the first factor that has something to do with the language is on the 6th place. Two other languages are on the 8th and 12th place. http://www.eecs.berkeley.edu/~lmeyerov/projects/socioplt/viz/index.html


Slide 16

What correlates most with enjoyment? Meyerovich and Rabkin also looked into what makes programmers happy. Want to guess?


Slide 17

Expressiveness Meyerovich and Rabkin also looked into what makes programmers happy. Want to guess? It’s expressiveness. That correlates with enjoyment most.


Slide 18

Could we measure it? Meyerovich and Rabkin also looked into what makes programmers happy. Want to guess? It’s expressiveness. That correlates with enjoyment most. So what language is most expressive? Is there a way to measure this?


Slide 19

Danny Berkholz, researcher at RedMonk came up with a way to do this. He compared commit sizes of different projects (from Ohloh, covering 7.5 million project-months) His assumption is that a commit has more or less the same ‘value’ in terms of functionality over different languages. http://redmonk.com/dberkholz/2013/03/25/programming-languages-ranked-by-expressiveness/


Slide 20

In the graph, the thick line indicates the median, the box represents 25 and 75% of the values and the lines 10 and 90%. http://redmonk.com/dberkholz/2013/03/25/programming-languages-ranked-by-expressiveness/


Slide 21


Slide 22

In the graph, the thick line indicates the median, the box represents 25 and 75% of the values and the lines 10 and 90%. Let’s have a look at what language goes where! http://redmonk.com/dberkholz/2013/03/25/programming-languages-ranked-by-expressiveness/


Slide 23

Loads of interesting things to see here. For instance, all the popular languages (in red) are on the low end of expressiveness. This somehow corroborates Meyerovich findings: while programmers enjoy expressiveness, it seems not to be a factor for picking a language.


Slide 24

Loads of interesting things to see here. For instance, all the popular languages (in red) are on the low end of expressiveness. This somehow corroborates Meyerovich findings: while programmers enjoy expressiveness, it seems not to be a factor for picking a language. Interesting is also the huge difference between CoffeeScipt and JavaScript, while this might be due to the fact that CoffeeScript is young and commits are thus quite ‘clean’.


Slide 25

Loads of interesting things to see here. For instance, all the popular languages (in red) are on the low end of expressiveness. This somehow corroborates Meyerovich findings: while programmers enjoy expressiveness, it seems not to be a factor for picking a language. Interesting is also the huge difference between CoffeeScipt and JavaScript, while this might be due to the fact that CoffeeScript is young and commits are thus quite ‘clean’. Finally, unsurprising, functional (Haskell, F#, Lisps) = expressiveness.


Slide 26

Another interesting fact came out of Meyerovich and Rabkin’s study.


Slide 27

Programmers don’t really believe in static typing


Slide 28

Programmers don’t really believe in static typing


Slide 29

Programmers don’t really believe in static typing


Slide 30

Could we measure it?


Slide 31

Stefan Hanenberg Static versus dynamic, does it really matter? Let’s experiment! Stefan Hanenberg tried to measure whether static typing has any benefits over dynamic typing.


Slide 32

Two groups Stefan Hanenberg tried to measure whether static typing has any benefits over dynamic typing. He divided a group of students into two groups, one with a type system and one without, and had them perform small maintainability tasks. Let’s summarize his results... Star Wars style!


Slide 33

Dynamic lover Legendary opponent On the left, we have the lover of dynamically typed stuff. He has gone through some ‘type casts’ in his life and he is sick of it! On the right is his static opponent. Let’s see how this turns out.


Slide 34

But what about type casting? One of the arguments that dynamic proponents have, is that type casting is annoying and time consuming. Hanenberg’s experiment showed:


Slide 35

It does not really matter But what about type casting? One of the arguments that dynamic proponents have, is that type casting is annoying and time consuming. Hanenberg’s experiment showed: It does not really matter. For programs over 10 LOC, you are not slower if you have to do type casting.


Slide 36

I’m sure I can fix type errors just as quickly


Slide 37

I’m sure I can fix type errors just as quickly Not even close


Slide 38

I’m sure I can fix type errors just as quickly Not even close The differences are HUGE! Blue bar = Groovy, Green = Java Vertical axis = time


Slide 39

I’m sure I can fix type errors just as quickly Not even close The differences are HUGE! Blue bar = Groovy, Green = Java Vertical axis = time In some cases, the run-time errors occurred at the same line where the compiler found a type error.


Slide 40

But… my dynamically typed APIs, they must be quicker to use


Slide 41

But… my dynamically typed APIs, they must be quicker to use No


Slide 42

I’ll document the APIs!


Slide 43

I’ll document the APIs! Won’t help! Turns out, type names help more than documentation.


Slide 44

I’ll use a better IDE!


Slide 45

Won’t help! I’ll use a better IDE!


Slide 46

Stefan Hanenberg It looks like (Java-like) static type systems really help in development! While more research is needed, we might conclude this.


Slide 47

Do design patterns work? Let’s tackle another one!


Slide 48

Walter Tichy Let’s tackle another one! Walter Tichy wanted to know whether design patterns really help development. He started small, with a group of students, testing whether giving them info on design patterns helped understandability.


Slide 49

Again two groups, but different Let’s tackle another one! Walter Tichy wanted to know whether design patterns really help development. He started small, with a group of students, testing whether giving them info on design patterns helped understandability. Again, students were divided into two groups, but setup was a bit different.


Slide 50

There were two programs used (PH and AOT) and some students got with the version documentation first and without second. On different programs obviously, otherwise the students would know the patterns were there in the second test. Prechelt et al, 2002. Two controlled experiments assessing the usefulness of design pattern documentation in program maintenance. TSE 28(6): 595-606


Slide 51

The results clearly show that knowing a pattern is there, helps performing maintenance tasks. However, it was not entirely fair to measure time, as not all solutions were correct. If you look at the best solutions, you see a clear difference in favor of the documented version. Ticky updated the study design in the next version, where only correct solutions were taken into account.


Slide 52

Again two groups, but again different The results clearly show that knowing a pattern is there, helps performing maintenance tasks. However, it was not entirely fair to measure time, as not all solutions were correct. If you look at the best solutions, you see a clear difference in favor of the documented version. Ticky updated the study design in the next version, where only correct solutions were taken into account.


Slide 53

In this next version, professionals were used instead of students. Furthermore, the setup was different. The same experiment was done twice, first without participants knowing patterns. Then, they did a course and after that again they did a test.


Slide 54

Again, results showed that version with patterns turned out to be easier to modify.


Slide 55

Again, results showed that version with patterns tuned out to be easier to modify. For some patterns though (like Observer) the differences in pre- and posttest were really big. For these patterns, the course made a big difference. In other words: patterns only help if you understand them.


Slide 56

They work! But why?


Slide 57

It has to do with how the human brain works


Slide 58

Long term memory Short term memory The human memory works a bit like a computer. Long term memory can save stuff for a long time, but it is slow. Short term memory is quick, but can only retain about 7 items. Using patterns, you only use 1 slot “this is an observer pattern” rather than multiple for “this class is notified when something happens in this other class” "The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information“, George Miller, 1956


Slide 59

They work! Experiments show it, cognitive science helps us understand why


Slide 60

Does Linus’ Law’ hold?


Slide 61

Given enough eyeballs, all bugs are shallow Does Linus’ Law’ hold?


Slide 62

Appearently, the law is not so universal, we know that code reviews are hard to do right for larger pieces of software.


Slide 63

Not impressed? Appearently, the law is not so universal, we know that code reviews are hard to do right for larger pieces of software. Yeah, a tweet is not exactly science. Don’t worry, I have some proof too.


Slide 64

Researcher at Microsoft research published a study in which they connected the number of bugs in the release of Vista (gathered through bug report) with organizational metrics, like the number of people that worked on a particular binary. Nagappan et al, 2008. The Influence of Organizational Structure On Software Quality: An Empirical Case Study, ICSE 2008


Slide 65

They found that the opposite of Linus’ law is true. The more people work on a piece of code, the more error-prone it is! Nagappan et al, 2008. The Influence of Organizational Structure On Software Quality: An Empirical Case Study, ICSE 2008 More touchers -> more bugs


Slide 66

More tied to bugs than any code metric They found that the opposite of Linus’ law is true. The more people work on a piece of code, the more error-prone it is! These, and other, organizational big are more tied to quality than any other code metric! Nagappan et al, 2008. The Influence of Organizational Structure On Software Quality: An Empirical Case Study, ICSE 2008


Slide 67

They found that the opposite of Linus’ law is true. The more people work on a piece of code, the more error-prone it is! These, and other, organizational big are more tied to quality than any other code metric! This means that if you want to predict future defects, the best you can do is look at the company! Might be me, but I think that is surprising. Nagappan et al, 2008. The Influence of Organizational Structure On Software Quality: An Empirical Case Study, ICSE 2008


Slide 68

Putting the science in computer science Felienne Delft University of Technology (@felienne) That’s it! I hope you got a sense for the usefullness of software engineering research in practice. If you want to keep up, follow my blog where I regularly blog about the newest SE research.


×

HTML:





Ссылка: