.TL .sp .nf On Performance .ig .br .ps -12 And the Future of Programming .ps .. .AU .ft R .sp 1c .ps 10 \fImarkus schnalke \fP .LP .S "Contents .LP Some mediations on software performance .I Case studies .I How to and how not to optimize .I A different view angle on performance .P "What is Performance? .S "Definition .QP Computer performance is characterized by the amount of useful work accomplished by a computer system compared to the time and resources used. .QP Depending on the context, good computer performance may involve one or more of the following: .QP .nf \(bu Short response time for a given piece of work \(bu High throughput (rate of processing work) \(bu Low utilization of computing resource(s) \(bu High availability of the computing system or application \(bu Fast (or highly compact) data compression and decompression \(bu High bandwidth / short data transmission time .sp \(em Wikipedia: Computer performance .P "The Useless Use of Cat Award .S "Jargon File: UUOC .LP .ps -2 .vs -2 .nf \fI[from the comp.unix.shell group on Usenet]\fP Stands for Useless Use of cat; the reference is to the Unix command cat(1), not the feline animal. As received wisdom on comp.unix.shell observes, ``The purpose of cat is to concatenate (or `catenate') files. \fBIf it's only one file, concatenating it with nothing at all is a waste of time, and costs you a process\fP.'' Nevertheless one sees people doing .sp .5 .CW " cat file | some_command and its args ... .sp .5 instead of the equivalent and cheaper .sp .5 .CW " .sp 1.5 >>>>> "Tony" == Tony Nugent writes: .sp .5 Tony> \f(CWcat file | sed -e '/^==========/,$d' > part.1\fP .sp .5 Wow. This week's "\fBUseless Use of Cat Award\fP" is being handed out on the first day of the week. I can rest now. :-) .sp .5 Hint: whenever cat has one argument, or no arguments, it's not *concatenating* anything, and can probably be removed. .sp .5 In your case: .sp .5 \f(CWsed -e '/^==============,$d' >part.1 #1/1 .sp 1.5 This week's useless use of cat award goes to... .sp .5 >>>>> "S" == S Cowles [who] writes: .sp .5 S> An example of a simple command line restore job is: S> \f(CWcat work.dump | ufsrestore xvf - './work/ '\fP .sp .5 Which of course as most of you know by now should be written as: .sp .5 \f(CWufsrestore xvf - './work/ ' .sp 1.5 >>>>> "PLM" == Peter Mutsaers writes: .sp .5 PLM> In the bourne shell do PLM> \f(CWcat filename | while read line\fP PLM> \f(CWdo\fP PLM> . PLM> . PLM> \f(CWdone\fP .sp .5 Aha. The winner of this week's "\fBuseless use of cat\fP" award. .sp .5 Hint: nearly any time you have just *one* argument to cat, you *probably* don't need the cat. .sp .5 [...] .sp .5 Just another would-be shell programmer (if it weren't for Perl :-), .S "Why UUOC? .I Randal L. Schwartz is JAPH .I Perl's view of the world .sp .LP Historical background { .RS .I ``v6 shell'' aka. osh(1): separate glob(1), if(1), goto(1) .I Shell text processing: sh + cut + tr + sed + awk + ... .I slow fork(2); length limits \(-> Perl (1987) .RE .LP } .sp .I Perl's view of the world .I Randal L. Schwartz is JAPH .S "Why people do waste cats .LP Myth of the performance difference .sp .LP Mind model: Pipelines .I Data flow: source \(-> filter... \(-> sink .I Cat as a generic data source .I Syntax should emphasize the semantics .S "Syntax comparisons .LP .ft CW .nf cat a | foo | bar | ... foo a | bar | ... foo b foo b .sp 2 cat a | while read line ; do ... done DEMO (usage, code, storage) .S "Problem: Object serializing .I Human readability? .I Mixture of static and dynamic data .I Software leverage? .LP \(-> To get a feeling: Extensively modify the subscription list! .sp 2 .QP Warning: The pickle module is not intended to be secure against erroneous or maliciously constructed data. Never unpickle data received from an untrusted or unauthenticated source. .br \(em Python Docs: The Python Standard Library .LP Internal data structure: Tied to some version? .sp .LP At least, it's plain text ... .S "Motivation .LP The most straight-forward implementation: .QP In the computer programming language Python, \fIpickle\fP is the standard mechanism for object serialization; .br \(em Wikipedia: Pickle (Python) .sp .LP Provided by the programming language: .QP The \fIpickle\fP module implements a fundamental, but powerful algorithm for serializing and de-serializing a Python object structure. .br \(em Python Docs: The Python Standard Library .sp .LP No explicit format conversions .LP .S "Beware .I ``Box thinking'': Just don't care about the outside world ... .I Contrast to Unix' toolchest approach: Write programs to work together! .I Care for the system-perspective! .ig \"--------------------- .P "Binary Protocols .S "Uzbl .LP A web browser!? .LP .nf Rather: Web interface tools which adhere to the unix philosophy .LP Had been a very promising project, in the beginning .S "Background .LP Timeline: .I yyyy-mm: uzbl idea in Archlinux forums .I yyyy-mm: first implementation by DieterBe .I yyyy-mm: my lightning talk at CCC Stuttgart .I yyyy-mm: DieterBe presents uzbl at ... .I yyyy-mm: DieterBe leafs the project .LP Then ... .S "Uzbl: Discussion 2011-06 .LP .ps -4 .vs -4 .nf To: uzbl-dev@lists.uzbl.org Date: 2011-06-14 01:00 From: Paul Colomiets Subject: [uzbl-dev] Binary protocol Message-ID: .sp 1.5 Hi, .sp .5 \fBI think that uzbl is now mature enought to get it's own binary protocol :)\fP .sp .5 Current protocol implementation relies on "EVENT" on start of line, and "\\n" at the end. Other tools are even more fragile, [...] .sp .5 So we should define a protocol which solves the following problems: .sp .5 1. Delimit messages and parts (instance name, event name, arguments...) 2. Separate stream of events from stream of RPC 3. Match remote procedure calls to replies in asynchronous way .sp .5 [...] .sp .5 It would also remove dependency of qtile on socat. .S "Consequences .LP Violation of one core aspect of the Unix philosophy: .QP Write programs to handle text streams, because that is a universal interface. .br \(em Doug McIlroy .LP \&... in a software project that tries to adhere to the Unix philosophy! .sp .LP No more universal combination with Unix tools! .sp .LP I unsubscribed the ML, disappointed .. \"--------------------- .P "MH's m_getfld .S "MH (nmh, mmh) .I Mail Handling tools .I Originate from the late 70s .I Heavily developed in the 80s .I Still used ... ;-) .sp 2 .LP sbr/m_getfld.c:\0 read/parse an RFC 822 message .S "Comments on m_getfld .QP And it seems that while all roads lead to Rome, all data in nmh goes through m_getfld() at some point. And that's where the fun begins ... the function is LITERALLY cursed! :-) .br \(em Ken Hornstein (2012) .sp 2 .QP /* This module has a long and checkered history. */ .sp 2 .QP my thought is, fire photon torpedoes. m_getfld was the wrong approach when it was new but it worked well enough (especially on slower older machines). .br \(em Paul Vixie (2012) .S "Reasons ... .LP Van Jacobson (1986): .QP This routine was accounting for 60% of the cpu time used by most mh programs. I spent a bit of time tuning and it now accounts for <10% of the time used. .sp .5 \&... .S "... and problems .QP \&... .sp .5 Like any heavily tuned routine, it's a bit complex and you want to be sure you understand everything that it's doing before you start hacking on it. Let me try to emphasize that: every line in this atrocity depends on every other line, sometimes in subtle ways. You should understand it all, in detail, before trying to change any part. If you do change it, test the result thoroughly [...]. "Minor" bugs in this routine result in garbaged or lost mail. .sp .5 If you hack on this and slow it down, I, my children and my children's children will curse you. .S "A closer look .LP .ps -2 .vs -2 i've just looked at m_getfld.c, for the first time since 1994 or so. \fBthis was good code in the pdp11 era\fP. .sp .5 this is a stateful iterator which does character level processing from an underlying stdio FILE object. \fBits caller must know internal details\fP of the state machine. opaqueness is nowhere attempted. \fBit digs into the underlying FILE object\fP to effect multi-character "ungetc" which is not supported by POSIX stdio, and it also \fBreturns pointers into the underlying FILE object's buffer\fP to avoid character copying, \fBall with #ifdef's\fP for LINUX_STDIO which presumably works differently. it tries hard to give the compiler hints to use the vax MATCH3 instruction for substring searching. its API and its implementation make UTF-8 impossible and by the time this thing has returned it is not possible for the caller to perform any I18N processing on fields like "subject" that can have same. its indentation has several off-by-one shifts. .sp .5 those of you who knew me in 1990 know that i used to write code like this; it was an art; \fBm_getfld.c is high art\fP. .br \(em Paul Vixie (2012) .S "Optimiziation .I 1) Good performance is good .I 2) Bad performance needs optimization! .LP \&... really? .sp .LP As everyone knows: .QP Premature optimization is the root of all evil. .br \(em Donald Knuth .sp .LP Optimization is a compromise .LP We're just ranking the various goals .S "How to optimize well .LP Rarely! .LP After having identified the real problem sources! .LP As a temporary crutch! .sp .QP Okay, the speed concerns mattered a lot back on a VAX; I think everyone agrees that nowadays it's not a big deal. .br \(em Ken Hornstein (2012) .P "The User .S "A broader view on Performance .LP Computer-oriented performance: .I Runtime performance .I Space performance .sp .LP User-oriented performance: .I Presentation performance .I Debugging performance .I Maintenance performance .I Code reading performance .I \&... .S "Changes .LP Back then: .I Computer time and space were expensive .I Users invested work to save computer work .sp .LP Today: .I Computer time and space are cheap .I Computers should work to reduce \fBour\fP work .sp .LP In consequence: .I Ignore computer performance optimizations .I \fBFocus on user-oriented performance optimization .S "Summary .I Syntax should reflect the mind model (semantics) .I Don't box think! Enable software leverage! .I View computer performance optimizations as temporary crutches! .I Care about humans, not about computers! .P "References .S Literature .nr PS -2 .nr PD 4p .I \f(CWhttp://en.wikipedia.org/wiki/Computer_performance .sp .5 .I \f(CWhttp://catb.org/jargon/html/U/UUOC.html .I \f(CWhttp://partmaps.org/era/unix/award.html .I \f(CWhttp://www.in-ulm.de/~mascheck/various/uuoc/ .sp .5 .I \f(CWhttp://en.wikipedia.org/wiki/Thompson_shell .I \f(CWhttp://v6shell.org/ .I \f(CWhttp://www.in-ulm.de/~mascheck/bourne/#predecessors .sp .5 .I \f(CWhttp://www.allthingsrss.com/rss2email .I \f(CWhttp://docs.python.org/2/library/pickle.html .I \f(CWhttp://en.wikipedia.org/wiki/Pickle_(Python) .sp .5 .I \f(CWhttp://marmaro.de/prog/mmh .I \f(CWhttp://git.marmaro.de/?p=mmh;a=blob;f=docs/m_getfld.c.humor .nr PD 12p .nr PS +2 .S .LP This talk was prepared using tools of the Heirloom project: .CW \s-2http://heirloom.sf.net\s+2 .LP The slides macros are based on .br .CW \s-2http://repo.cat-v.org/troff-slider/\s+2 .sp .LP The slides are available on my website .CW \s-2http://marmaro.de/docs/\s+2 .sp .LP .sp |12c 2013-11-11 at CCC Ulm, ChaosSeminar