It turns out to be very easy to implement an async servlet (new in servlet 3.0). In Scala, it took less than 10 lines of code (not counting brace brackets and imports):

package com.ttdev.webqos
import javax.servlet.http._
import java.util.concurrent._

class MyServlet extends HttpServlet {
  private val pool = Executors.newFixedThreadPool(1) // create a thread pool

  override def doGet(req: HttpServletRequest, res: HttpServletResponse) {
    req.startAsync  //tel the container the response will be generated later
    pool.execute(new Runnable() {  // add the runnable to the queue
      def run {  // when a thread in pool picks up the runntable...
        res.getWriter.append("hello world!")  // write the response
        req.getAsyncContext.complete  // tell the container that the response is done
      }
    })
    // return without a response yet
  }

}

Many people have different interpretations of what software architecture actually means. Some people take it as the high level decomposition of the system (subsystems or physical distribution). However, a much better interpretation is simply “the most important design decisions”. What are important design decisions? If they are going to impact what you (or your customer) care the most, then it is important. Typically, if the design decision is going to impact the availability, performance, scalability, security, usability, manageability, maintainability, cost, then it is part of the architecture.

Examples

Below are some examples of architectural decisions:

• Availability: how to build in redundancy in all the layers (storage, middle tier) and how to fail-over (e.g., client initiated vs virtual IP)? How to handle excessive load (e.g., QoS)? Selecting well proven products with strong support?
• Scalability: how to scale (scale out or scale up) and load balance?
• Performance: multi-threading, algorithm, processing in batches?
• Security: encrypting data against exposure, using HMAC to check data integrity, secure communication, auditing?
• Manageability: monitoring and operational control (e.g., logging, JMX)? Live upgrading (OK to upgrade just one node at a time in a cluster)?
• Maintainability: automated tests, using a language, tools and frameworks familiar to the team, proper layering of the code (UI, service, data access layers)?
• Cost: language, tools and frameworks familiar to the team, open source vs commercial products?

Specifying and maintaining the architecture

I find the best way to specify the architecture is to explain it in use cases (e.g., how to perform fail-over when a certain component fails or how to ensure security in a typical use case).

Also, specifying the architecture is just the first step. Some team members may make sub-optimal architectural decisions in programming without knowing it, or you may realize that your initial decisions can be further improved when you see the code. Therefore, as the architect, your work is far from over and you must keep the architecture (as implemented in the code) fit as the project goes.

Just watched Howard Lewis Ship’s Clojure: Towards The Essence Of Programming, which is an excellent introduction to Clojure and how it addresses the problems in Java. As a Scala fan, I thought it would be a good idea to see how one could use Scala to do the same.

Ceremony vs Essence

Howard uses the example below to show how much ceremony (code with little real info) is required, hiding the essence (the real intention) when using Java to sort a list of Stock objects (a portfolio) either using different properties such as open values or last trade values:

 public static void sortByOpen(List<Stock> portfolio) {
    //ceremony: a lot of code but very little real info
    Comparator<Stock> c = new Comparator<Stock>() {
       public int compare(Stock o1, Stock o2) {
          return o1.getOpen() - o2.getOpen(); //essence: the beef is here
       }
    };
    Collections.sort(portfolio, c);
}

The Clojure version is:

//a Stock object is just a map with each field as a key. ":open" is the key to get the open value.
(sort-by :open portfolio)

In Scala, it can be as simple as the Clojure version:

//full version: using an anonymous function instead of a comparator object
portfolio.sortBy((s: Stock) => s.open)

//simpified version: sortBy is expecting the argument to be a function
// (Stock) => SOME-RESULT-TYPE, so we don't need to declare the
//type for "s".
portfolio.sortBy((s) => s.open)

//final version: as "s" only appears once in the function body, just use
//an underscore and get rid of the parameter declaration.
portfolio.sortBy(_.open)

The Clojure version is like the final Scala version in the first attempt because it doesn’t need typing information in the code. In Scala, the compiler infer it from the code.

In addition, in Clojure the object is just a map so people can refer to a property directly with its name without referring to the object. In Scala, we must use something to denote the object in the form of obj.property. However, the underscore syntax eliminates the need to declare that object, so the code is still very succinct.

The benefits of the Scala approach are:

• Compile time checking: if the property name mismatches the Stock class, in the Scala the compiler will tell us immediately.
• Runtime performance: it is a lot faster to access the property by offset than by looking up a key.

The cost we pay is mainly the learning curve: we must learn how Scala infers the typing information so that we know what to type and what to NOT type and learn the meaning of the underscore in an anonymous function.

Form is structure

I may not be fully understanding this example. Howard uses the Fahrenheit conversion, f =9*c/5+32, to show that in Clojure the form (the look on the surface) is the same as the structure (the actual relationship):

//the look is the same as the actual syntax tree
(+
  (/
    (* 9 c)
    5)
  32)

I definitely think the Java or Scala version is much simpler and more concise:

9*c/5+32

This is, after all, what we call DSL. Arithmetics has its rules of precedence and associativity, allowing us to write in a very succinct form to express more complex structure.

Read Eval Print Loop

Scala has it too. I like it very much too.

Functional programming

Howard that moves onto functional programming with examples using filter, reduce, map, for comprehension, stream. Obviously, Scala can do all these in a similar fashion. For example, to get the last trade total value of the portfolio, we can map each stock to its last trade value and then use reduce to sum them all up. In Clojure:

//#() means anonymous function. % refers to the one and only parameter (the stock).
(reduce + (map #(* (% :last-trade) (% :shares)) portfolio))

In Scala:

//+ in Scala is not a function, but a method. So we can't use it directly;
//instead, use an anonymous function with underscores again.
portfolio.map((s)=>s.lastTrade*s.shares).reduceLeft(_+_)

There is not much difference. However, I think the significance is much deeper. The question is not whether functional programming is supported or not, but how much it is intended to be used in place of procedural programming. Scala allows both styles of programming, which may or may not be a good thing, depending on how you view it. If you’re committed to the functional programming style for concurrency, testability, predictability, then using a language that is less capable of procedural programming is actually a good thing. If you’re more comfortable with procedural programming and just using functional programming in a smaller scale, then a hybrid language would be better.

Extending the language with DSL

Then Howard moves onto extending Clojure with DSL. One example he uses is building a DOM tree with a DSL.

(defview root-index
  [env]
  :html [
    :head [
      :title [ "Cascade Blog" ]
    ]
    :body [
      :h1 [ "Cascade Blog" ]
      :ul { :class "recent-postings" } [
        (template-for [posting (recent-postings env)]
          :li [
             (render-link env show-posting (posting :id) (posting :title))
           ])
      ]
    ]
  ])

As an exercise I implemented a similar DOM DSL in Scala:

//the DSL is implemented in the Html object
import Html._

//using the DSL to define a DOM tree
p @@ ("style" -> "color: red") containing (
        "This is an ",
        b containing ("interesting"),
        "DOM tree"
        )

They aren’t that different except that in Clojure each list item is separate from its neighbors by being enclosed in its own () but in Scala we must use a comma to separate the list items. So, choose your own poison: either tolerate a million ()’s or the ugly commas in the DSL.

Another difference is that in Clojure text can be freely mixed in the DOM tree as there is no typing restriction, while in Scala text is magically converted into a TextNode object. Again, the latter provides structure, compile-time protection and runtime performance, but we must learn the implicit conversion mechanism to understand what is going on.

Just to be complete, the DOM DSL is implemented in Scala as below:

//a DOM node
class Node

//a text node
case class TextNode(val text: String) extends Node

//an element node
case class Element(val name: String,
                   val attrs: Map[String, String],
                   val children: Seq[Node]) extends Node {
  def this(name: String) = this (name, Map(), List())

  //we can use symbols as method names, but @ is a reserved keyword in Scala,
  //so I used @@ as the name of the method that adds attributes to the element.
  def @@(attrPairs: (String, String)*) =
    //just add each pair to the map
    Element(name, attrPairs.foldLeft[Map[String, String]](Map())(_ + _), children)

  //add the child nodes (elements or text nodes) to this element
  def containing(children: Node*) = Element(name, attrs, children)
}

object Html {
  //convert strings into text nodes automatically
  implicit def string2TextNode(s: String) = TextNode(s)
  def p = new Element("p")
  def b = new Element("b")
}

Howard also shows the need for lazy evaluation so that some code is only executed on demand. For example, the DSL code to generate each <li> is repeatedly called in a loop. In Clojure lazy evaluation is implemented with macros which generate well-form code instead of arbitrary text. This seems to be more powerful (able to do more) than the call-by-name mechanism Scala, although I don’t know if/when this extra power is really needed.

A method can appear in an expression as an internal value (to be called with arguments) but it can’t be the final value, while a function can:

//a simple method
scala> def m(x: Int) = 2*x
m: (x: Int)Int

//a simple function
scala> val f = (x: Int) => 2*x
f: (Int) => Int = <function1>

//a method can't be the final value
scala> m
<console>:6: error: missing arguments for method m in object $iw;
follow this method with `_' if you want to treat it as a partially applied function
       m
       ^

//a function can be the final value
scala> f
res11: (Int) => Int = <function1>

Parameter list is optional for methods but mandatory for functions

A method can have no parameter list or have one (empty or not), but a function must have one (empty or not):

//a method can have no parameter list
scala> def m1 = 100
m1: Int

//a method can have an empty parameter list
scala> def m2() = 100
m2: ()Int

//a function must have a parameter list
scala> val f1 = => 100
<console>:1: error: illegal start of simple expression
       val f1 = => 100
                ^
//a function's parameter list could be empty
scala> val f2 = () => 100
f2: () => Int = <function0>

Why a method can have no parameter list? See below.

Method name means invocation while function name means the function itself

Because methods can’t be the final value of an expression, so if you write a method name and if it doesn’t take any argument (no argument list or an empty argument list), the expression is meant to call that method to get the final value. Because functions can be the final value, if you just write the function name, no invocation will occur and you will get the function as the final value. To force the invocation, you must write ():

//it doesn't have a parameter list
scala> m1
res25: Int = 100

//it has an empty parameter list
scala> m2
res26: Int = 100

//get the function itself as the value. No invocation.
scala> f2
res27: () => Int = <function0>

//invoke the function
scala> f2()
res28: Int = 100

Why we can provide a method when a function is expected?

Many Scala methods such as map() and filter() take functions arguments, but why can we provide methods to them like:

scala> val myList = List(3, 56, 1, 4, 72)
myList: List[Int] = List(3, 56, 1, 4, 72)

//the argument is a function
scala> myList.map((x)=>2*x)
res29: List[Int] = List(6, 112, 2, 8, 144)

//try to pass a method as the argument instead
scala> def m3(x: Int) = 3*x
m3: (x: Int)Int

//still works
scala> myList.map(m3)
res30: List[Int] = List(9, 168, 3, 12, 216)

This is because when a function is expected but a method is provided, it will be automatically converted into a function. This is called the ETA expansion. This makes it a lot easier to use the methods we created. You can verify this behavior with the tests below:

//expecting a function
scala> val f3: (Int)=>Int = m3
f3: (Int) => Int = <function1>

//not expecting a function, so the method won't be converted.
scala> val v3 = m3
<console>:5: error: missing arguments for method m3 in object $iw;
follow this method with `_' if you want to treat it as a partially applied function
       val v3 = m3
                ^

With this automatic conversion, we can write concise code like:

//10.< is interpreted as obj.method so is still a method. Then it is converted to a function.
scala> myList.filter(10.<)
res31: List[Int] = List(56, 72)

Because in Scala operators are interpreted as methods:

• prefix: op obj is interpreted as obj.op.
• infix: obj1 op obj2 is interpreted as obj1.op(obj2).
• postfix: obj op is interpreted as obj.op.

You could write 10< instead of 10.<:

scala> myList.filter(10<)
res33: List[Int] = List(56, 72)

How to force a method to become a function?

When a function is not expected, you can still explicitly convert a method into a function (ETA expansion) by writing an underscore after the method name:

scala> def m4(x: Int) = 4*x
m4: (x: Int)Int

//explicitly convert the method into a function
scala> val f4 = m4 _
f4: (Int) => Int = <function1>

scala> f4(2)
res34: Int = 8

A call by name parameter is just a method

A call by name parameter is just a method without a parameter list. That's why you can invoke it by writing its name without using ():

//use "x" twice, meaning that the method is invoked twice.
scala> def m1(x: => Int) = List(x, x)
m1: (x: => Int)List[Int]

scala> import util.Random
import util.Random

scala> val r = new Random()
r: scala.util.Random = scala.util.Random@ad662c

//as the method is invoked twice, the two values are different.
scala> m1(r.nextInt)
res37: List[Int] = List(1317293255, 1268355315)

If you "cache" the method in the body, you'll cache the value:

//cache the method into y
scala> def m1(x: => Int) = { val y=x; List(y, y) }
m1: (x: => Int)List[Int]

//get the same values
scala> m1(r.nextInt)
res38: List[Int] = List(-527844076, -527844076)

Is it possible to maintain the dynamic nature of x in the body? You could cache it as a function by explicitly converting it:

//explicit conversion, but then you must invoke the function with ().
scala> def m1(x: => Int) = { val y=x _; List(y(), y()) }
m1: (x: => Int)List[Int]

scala> m1(r.nextInt)
res39: List[Int] = List(1413818885, 958861293)

The problem with equals() is that it is difficult to get it right. For example, for a simple class Foo, you may write the equals() method as:

class Foo(val i: Int) {
  override def equals(that: Any) = {
    that match {
      case f: Foo => f.i == i
      case _ => false
    }
  }
}

The problem is that what happens if the other object (“that”) belongs to a subclass of Foo, which may have its own fields? As the equals() method is only comparing the “i” field, it will ignore the other fields and return true prematurely. Below is such a subclass Bar:

class Bar(i: Int, val j: Int) extends Foo(i) {
  override def equals(that: Any) = {
    that match {
      case b: Bar => super.equals(b) && b.j == j
      case _ => false
    }
  }
}

With the erroneous equals() method in Foo, we could get incorrect results:

scala> val f1 = new Foo(2)
f1: Foo = Foo@14c0275

scala> val b1 = new Bar(2, 3)
b1: Bar = Bar@171bc3f

scala> f1.equals(b1)
res0: Boolean = true

scala> b1.equals(f1)
res1: Boolean = false

A solution to the problem

The problem is that the equals() method in Foo is treating the Bar object exactly as a base Foo object, but the equality contract in Bar has changed from that in Foo. Of course, not every subclass of Foo will use a different equality contract; some do and some don’t (by default, we should assume that they don’t). Therefore, the equals() method in Foo should make sure that the “that” object uses the same equality contract as “this”:

object FooEqualityContract {
}

class Foo(val i: Int) {
  //by default all Foo objects and subclass objects use this equality contract
  val equalityContract: Any = FooEqualityContract

  override def equals(that: Any) = {
    that match {
      //make sure the two objects are using the same equality contract
      case f: Foo => f.equalityContract == this.equalityContract && f.i == i
      case _ => false
    }
  }
}

Now, as Bar is using its own equality contract, it should say so:

class Bar(i: Int, val j: Int) extends Foo(i) {
  //tell others that we're using our own equality contract
  override val equalityContract: Any = BarEqualityContract

  override def equals(that: Any) = {
    that match {
      case b: Bar => super.equals(b) && b.j == j
      case _ => false
    }
  }
}

Now, the equals() method in Foo will rightly determine that a Bar object is using a different equality contract and thus will never be equal to a bare Foo object:

scala> val f1 = new Foo(2)
f1: Foo = Foo@34b350

scala> val b1 = new Bar(2, 3)
b1: Bar = Bar@7c28c

scala> f1.equals(b1)
res2: Boolean = false

scala> b1.equals(f1)
res3: Boolean = false

scala> val b2 = new Bar(2, 3)
b2: Bar = Bar@5dd915

scala> b1.equals(b2)
res6: Boolean = true

Of course, it should also work for subclasses that use the same equality contract:

scala> val f2 = new Foo(2) { }
f2: Foo = $anon$1@a594e1

scala> f1.equals(f2)
res9: Boolean = true

scala> f2.equals(f1)
res10: Boolean = true

To fix the problem, there is a very simple solution: enhance the DNS server to perform a health check against the resulting host of the resource record. For example, the administrator could specify the TCP port to connect to as in the imaginary syntax below:

  www        A      1.1.1.1    80     ; return this record only if we can connect to its TCP port 80
  www        A      1.1.1.2    80
  www        A      1.1.1.3    80

Of course, the health check could be more general, then you could use a script:

  www        A      1.1.1.1    web-check.sh  ; return this record only if the script returns true

where the IP would be passed to that script as an argument for checking.

It works for domain controllers too:

  _ldap._tcp.dc._msdcs.foo.com.   SRV  1.1.1.1  dc-check.sh
  _ldap._tcp.dc._msdcs.foo.com.   SRV  1.1.1.2  dc-check.sh

Finally, one might ask why implement this checking in the DNS server instead of the clients? The idea is that problems should be detected as early as possible to avoid bad effects downstream. In concrete terms, if a server is down but the DNS server (broker) still refers the clients to it, many clients will need to perform this health check themselves. But if the DNS server performs this health check, the checking is only done once, saving a lot of trouble downstream.

Why some of expressions below work (can be evaluated and printed) but the other don’t?

Math.min  //doesn't work
Math.min _  //works
val f: (Int, Int)=>Int = Math.min  //works

The first expression doesn’t work because Math.min is a method, but a method is not a value in Scala. The second expression works because the underscore asks Scala to convert the method to a function, which is indeed a value in Scala. The third expression also works because when Scala is expecting a function value from the expression but finds a method, it will convert it to a function automatically.

Why some of expressions below work but the other don’t?

List(2, 3, 5) foreach println  //works
List(2, 3, 5) foreach println(_) //doesn't work
List(2, 3, 5) foreach (println(_)) //works

The first one works because foreach is expecting a function, while println (of the Predef object which has been imported automatically) is a method, not a function, but as mentioned above Scala will convert it into a function automatically because a function is expected. So it works.

The second one is trying to create an anonymous function:

List(2, 3, 5) foreach ((x)=>println(x))

However, in Scala only a “top level” expression can be a function. In this case, println(_) is only a “term” in an expression (foreach is the operator) but not a top level expression, so the compiler won’t try to make it an anonymous function. Instead, it will search further to find the first enclosing top level expression (in this case, the whole expression you entered) and turn it into an anonymous function:

(x)=>List(2, 3, 5) foreach println(x)

But then the type of x can’t be inferred, so it is an error. Also, println(x) returns a value of (), the only value of the class Unit, which is not what foreach wants anyway (a function taking an Int).

With this knowledge, you can see why the third expression works:

List(2, 3, 5) foreach (println(_)) //works

This is because with the parentheses, a top level expression is expected, so Scala will make println(_) an anonymous function:

List(2, 3, 5) foreach ((x)=>println(x)) //works

Why some of expressions below work but the other don’t?

List(2, 3, 5) foreach (println("hi"+_))  //doesn't works
List(2, 3, 5) foreach (println "hi"+_)  //doesn't works
List(2, 3, 5) foreach (Predef println "hi"+_)  //works

The first expression doesn’t work because the first top level expression found is “hi”+_ due to the parentheses. So, Scala will treat it as:

List(2, 3, 5) foreach (println((x)=>"hi"+x))  //doesn't works

So you’re printing a function to the console and returning a unit value () to foreach. In order to fix the problem, you may try to get rid of the parentheses so “hi”+_ is no longer a top level expression:

List(2, 3, 5) foreach (println "hi"+_)

The problem is that Scala will now try to parse:

println "hi"+_

as:

expr1 op1 expr2 op2 ...

println is assumed to be an expression instead of a prefix operator because only !, ~, +, – can be prefix operators. So, println will be treated as an expression while the String “hi” will be treated as an operator, which is obviously incorrect.

To fix this problem, you can provide a real object as expr1, which is the Predef object:

List(2, 3, 5) foreach (Predef println "hi"+_)

Note that there are two operators: println and +. Because all symbolic operators have higher precedence than identifier operators, + will be applied first.

So, the first enclosing top level expression is turned into an anonymous function:

List(2, 3, 5) foreach ((x)=>Predef println "hi"+x)

Because in Scala “e1 op e2″ is treated as e1.op(e2), the code is treated as:

List(2, 3, 5) foreach ((x)=>Predef.println("hi".+(x)))

For example, you’d like to learn about the Seq trait in Scala. So you open the API page of it and find a long list of methods. Let’s say you’re interested in learning the behaviors of the following methods:

/** Appends all elements of this sequence to a string builder. The written text consists of the string representations (w.r.t. the method toString) of all elements of this sequence without any separator string. */
def addString(b: StringBuilder): StringBuilder

To do that, you issue the “scala” command from a shell/command prompt and then explore the method like:

kent@dragon:~$ scala
Welcome to Scala version 2.8.0.r20327-b20091231020112 (Java HotSpot(TM) Server VM, Java 1.6.0_20).
Type in expressions to have them evaluated.
Type :help for more information.

scala> val a = List("a", "b", "c")
a: List[Int] = List(a, b, c)

scala> val b = new StringBuilder
b: StringBuilder = 

scala> b.append("hi")
res0: StringBuilder = hi

scala> a.addString(b)
res1: StringBuilder = hiabc

From the experiment, it is clear that the addString() method will append all the elements of the Seq to the string builder.

Let’s consider another. You’re interested in:

/** Multiplies up the elements of this collection. num is an implicit parameter defining a set of numeric operations which includes the * operator to be used in forming the product.
*/
def product[B >: A](num: Numeric[B]): B

So, try it in the Scala interpreter:

scala> val a = List(3, 5, 2)
a: List[Int] = List(3, 5, 2)

scala> a.product
res2: Int = 30

So it seems to work fine. Possible to override the * operator? From the API doc it is clear that the default being used is the IntIsIntegral object:

package scala.math

class Numeric {
  object IntIsIntegral extends IntIsIntegral with IntOrdering {
     ...
  }
}

So, the first try is:

scala> import scala.math.Numeric._
import scala.math.Numeric._

scala> val x = new IntIsIntegral {
     | override def times(x: Int, y: Int) = x+y;
     | }
<console>:9: error: object creation impossible, since method compare in trait Ordering of type (x: Int,y: Int)Int is not defined

Oops, forgot to specify the IntOrdering trait which defines the compare() method. So, do it now:

scala> import scala.math.Ordering._
import scala.math.Ordering._

scala> val x = new IntIsIntegral with IntOrdering {
     | override def times(x: Int, y: Int) = x+y;
     | }
x: java.lang.Object with math.Numeric.IntIsIntegral with math.Ordering.IntOrdering = $anon$1@1e5d007

We have successfully created a new object to redefine the * operator as just addition. Now, pass it to the product() method:

scala> a
res5: List[Int] = List(3, 5, 2)

scala> a.product(x)
res6: Int = 11

It should be 3+5+2=10 but why the result is 11? Recall that it is doing multiplication, so it is using 1 as the seed for calculation (1+3+5+2). To change the seed from 1 to, say, 0, we can override the one() method:

scala> val x = new IntIsIntegral with IntOrdering {
     | override def times(x: Int, y: Int) = x+y;
     | override def one = 0;
     | }
x: java.lang.Object with math.Numeric.IntIsIntegral with math.Ordering.IntOrdering = $anon$1@11a9f20

scala> a.product(x)
res7: Int = 10

Obviously now it works.

1. Late arrival to the office. Your boss is so blind to see your great contributions to the company, therefore you are so de-motivated that you come in to the office late everyday.
2. Micro-management. Your boss is always trying to micro-manage you by telling you the “right” ways to do things such as agile methodologies, but actually you know those aren’t just good as your way because you’ve been in the trenches long before your boss.
3. Rejecting your good suggestions. You have been pushing a complete conversion to a cutting edge technology to double the productivity of the team, but your boss just won’t listen.
4. Incompetent peers. Your peers are so incompetent and have been hampering the product launch. Even though you have been telling them their problems, they just don’t get it.
5. Being blamed. Your peers are so jealous of your great abilities that they try to isolate you and blame you for everything that has gone wrong.

These signs seemingly indicate that people aren’t appreciating your talents, but the fact is you may be just living in your own little world and will probably be fired in a few months! The truth that you may not be understanding right now is:

1. Late arrival to the office (Lack of commitment). Even if you aren’t happy with your job or the way you’re treated, you should still demonstrate your commitment to your duties. Arriving late is an obvious way to say that you have no commitment.
2. Micro-management (Not accepting advise for improvement). Seeing the difficulties you face, your boss is trying to help you by giving you good advice. But you are so attached to your ego and stubborn to see any values in any new approaches.
3. Rejecting your good suggestions (Not understanding the priorities of the company). Your suggested technology may seem great to yourself, but actually it may be just too premature or is not among the top priorities of the company. Everyone is trying to tell you that but you just won’t listen.
4. Incompetent peers (Destroying harmony). You may be good or not, but it is never a good idea to pick the mistakes of others. You will appear like an asshole to all your peers. Instead, you should help others improve.
5. Being blamed (Not seeing your own mistakes). Be brave and admit it, those mistakes were yours. If you don’t see your own mistakes, you’ll never grow.

Fortunately, it is not too late to realize your own mistakes. Stop denying and you’ll have a much brighter future.

ps, if you’d like to learn more about IT management and governance, check out IT Governance by Examples.

Then there was another client who spent a lot of time looking for and selecting a carpenter with good reputation for excellent customer satisfaction. He only told the carpenter that he needed a chair that is comfortable to sit on and then drafted a sketch of the chair. Then he negotiated with the carpenter to settle on the cost and duration. There was no reward for early delivery and no penalty for late delivery. They agreed to discuss to adjust the cost and duration if there turned out to be a need.

Then what was the result? The first client got the chair earlier than the deadline and the carpenter got the reward. However, when the client sat on the chair, it was not that comfortable. Because the carpenter rushed to finish it, the craftsmanship was poor and as a result, the chair was broken in a year. Then, it was found that low-graded wood was used in the inner, hidden part of the chair.

In contrast, the other client found that the chair was not that comfortable during the making. So he discussed with the carpenter to make the necessary change. The cost was increased by a little and the duration was longer by a little, but the carpenter had enough time for him to make a quality chair that he is proud of, which finally lasted for 10 years.

The moral of the story

The moral of the story is that in any project there are four factors to be considered: cost, time, scope and quality. If any of those changes, at least one of the other must change accordingly. The problem is, most people will try to fix/specify all those in planning, but in really only cost and time can be easily fixed, while it is very difficult to fix/specify the scope and quality (they are fuzzy and dynamic). So, from the view of the contractor, to meet the fixed cost and time, he has every incentive to minimize the scope (only make  what is explicitly told and nothing else) and reduce the quality (uncomfortable to sit, poor craftsmanship, low-graded wood).

The reward and the penalty make the matter worse. They double the importance of meeting the time factor, so they double the incentive for the contractor to minimize the scope and reduce the quality.

To solve this problem, many people try to make the scope and quality even better specified. However, this effort is futile as scope and quality are related to the needs of human which is fuzzy and dynamic. In the example of the chair, what is the quality? Perhaps that it should be comfortable to sit on, that it should last long, etc. But these aren’t objective (what is comfortable?) nor can be measured immediately (how long it will last?), so they can’t be used to bind the behavior of the contractor.

Therefore, there must be a second level of defense to deal with this fuzziness and dynamics. In addition to specifying the scope and quality at our best effort, we must also identify the contractor who really cares for quality. The idea is that he will work for quality on a higher-level (what we really care about) instead of focusing on meeting lower-level but objective metrics which do not necessarily fully reflect quality. That is the best insurance that we can get. Of course, to allow him to work for the fuzzy and dynamic scope and quality, we must be prepared to renegotiate with him to adjust the cost and duration as necessary.

ps, if you’d like to learn more about IT management and governance, check out IT Governance by Examples.