The year was 1995. I was working at FlexNet, a local ISP in The Woodlands (a northern suburb of Houston), as a tech support jockey and hack programmer. It was a small company, maybe 10 employees, and was owned by two old business partners who didn’t know much about the Internet but had other successful ventures in insurance and real estate. This was a great time to be involved with the Internet–it was still raw and rough around the edges but compared to today it was “pure”. The Internet still felt like an exclusive destination; something you could do that other people couldn’t. I still fondly remember the looks I used to get when I told other people, disconnected people, about the Internet–one part pity and two parts disgust.

This particular day was special. I was under our marketing guy’s desk installing a new video card in his new computer. He was ferociously pacing about, demanding that I hurry the hell up. As far as I was concerned this new video card had only one purpose: after hours Duke Nuke and Quake, something me and the 3 other “tech guys” at the ISP would do every weekend. However, Mike the Marketing Guy could care less. He wanted that video card installed and installed fast. Netscape was IPOing and he wanted to get online.

I didn’t grasp what was happening, but August 9, 1995 was the beginning of the great Internet/Tech bubble. Netscape really was the catalyst. It set the whole thing off. Netscape IPO’d at around $28/share and soared, ending the day up 100% at around $58/share giving it a market cap of nearly $3 billion. Netscape probably deserved most of this valuation; however, this isn’t about what Netscape deserved. This is about the greed that ensued.

You see, venture capitalists and entrepreneurs saw that any story connected to the Internet was a catalyst for very large valuations. A rush was on to IPO. Get in while the getting is good. People are paying, so we’re selling. This was the predominant viewpoint. Once the handful of deserving Internet companies had gone public the search was on for the next big thing. Before you knew it, companies were coming out with little to no revenue and demanding $1 billion valuations. The capital got more greedy than normal and the entrepreneurs were happy to oblige. Easy money.

Unfortunately, as we all know, it ended badly. We all learned our lesson, right?

LinkedIn IPO’d on Thursday May 19, 2011. It soared. Just like Netscape it probably deserved to soar. It’s been a stalwart in the social networking space and has a very large niche (professional social networks) all to its own. But again, just like Netscape, this isn’t about whether LinkedIn deserves $9 billion valuation it currently has. This is about what happens next.

Now that the primary markets are proven open for business to a new breed of dotcoms will we see a gradual but persistent decrease in common sense around the types of businesses that get funded and the types of exits they receive? We’re already hearing news that Zynga is pushing to IPO sooner rather than later–and again, Zynga is a very successful business and deserves to be public if they want to be. What you should be watching for is this: once the 6-12 deserving companies go public will we start to see less deserving businesses sell shares to the public? If so, you have a decision to make. Ride the wave or sit it out.

In my opinion, the next bubble is coming. These things don’t happen all at once nor do they end instantaneously. I’d reckon that if you interviewed any venture guy that lived through the bubble they would claim that they couldn’t get caught up in it again. However, everyone has penis envy. Everyone wants to see their portfolio have tremendous success and going public, especially in a frothy market, is a sure way to make that happen. Once the venture industry sees its possible I have a feeling they’ll begin to push companies to come out sooner than they should and entrepreneurs will be happy to oblige. Eventually we’ll see an erosion in quality of new issues and if valuations don’t adjust, then I’ll personally be staying very far away. Riding the wave can be great fun, but there’s rocks on this beach. Big, jagged, sharp net-worth-eating rocks.

Other posts you might like:

1. Onwards and Upwards: Goodbye Cisco, Hello TBD
2. Getting Things Done: An Entrepreneurial Prerequisite
3. Risk and Responsibility

This post is part of a series: Bash Tips and Tricks.

When I write a “serious” bash script I always try to check my pre-requisites. Is pigz installed? If not, fall back to gzip. Is wget installed? If not, error out. That sort of stuff. One part of this is checking the version of the programs you depend on. Most of the time version doesn’t matter, but every so often it does. There are some tools out there to do version comparisons, like Arch’s vercmp. However, I’ve never really found a ubiquitous tool that I could rely on being there, so I decided to write a bash function to do the comparison for me that relies only on sed, sort, and head–tools that are ubiquitous.

# return 0 if program version is equal or greater than check version
check_version()
{
    local version=$1 check=$2
    local winner=$(echo -e "$version\n$check" | sed '/^$/d' | sort -nr | head -1)
    [[ "$winner" = "$version" ]] && return 0
    return 1
}

The function, check_version, takes two arguments. The first is the version of the tool we are verifying and the second is the minimum version we’re willing to accept. Both version and check must only be major.minor version numbers, no patch levels, beta tags, or any other “version chrome”. The way it works is very simple:

1. Echo both versions one on each line, and send that as stdin to sed.
2. Eliminate empty lines with sed.
3. Use sort to do a reverse numeric sort.
4. Take the first line of sort‘s output, and call that the winner.
5. If the winner is the version from the tool we are verifying, the one passed in as argument 1, then the version of the tool checks out. Otherwise, it does not.

This function does exactly what I need, nothing more and nothing less. I’m sure it could be extended to support more than major/minor version numbers. If you do add features to it, please let me know.

Other posts you might like:

1. Showing a Bash Spinner for Long Running Tasks
2. Introducing Bash Tips and Tricks
3. Showing File Download Progress Using Wget

This post is part of a series: Bash Tips and Tricks.

I’ve mentioned before that I like to show users of my scripts useful information. This can be a complex progress bar, a simple progress meter, or it can be an animation to let the user know the script hasn’t hung. These animations are typically called Throbbers are exist purely to tell the user to continue patiently waiting. Sometimes folks like me call Throbbers Spinners for two reasons. First, a very common throbber animation type is the spinning wheel. Second, the word throbber sounds oddly sexual, and sort of creeps me out if I say it too many times in a sentence (Just kidding. Well, no, not really.)

This tip will show you to create a spinner for your Bash scripts. If you have a long running process and don’t want to try to tell the user approximately how much of that process is left to run, showing them a spinner is a great alternative.

Implementing the Spinner

The throbber, er I mean spinner, is implemented as a loop that shifts a string during each iteration.

spinner()
{
    local pid=$1
    local delay=0.75
    local spinstr='|/-\'
    while [ "$(ps a | awk '{print $1}' | grep $pid)" ]; do
        local temp=${spinstr#?}
        printf " [%c]  " "$spinstr"
        local spinstr=$temp${spinstr%"$temp"}
        sleep $delay
        printf "\b\b\b\b\b\b"
    done
    printf "    \b\b\b\b"
}

Let’s dissect the spinner function so to illustrate how it works.

local pid=$1
local delay=0.75
local spinstr='|/-\'

This is pretty self-explanatory. The function has one input argument and two internal variables that control how it works.

1. The input argument is assigned to the local variable pid. This is the process ID of the background task for which we are showing the spinner.

2. The next local variable, delay, is how long each frame of the spinner animation stays visible for. We’ve set it to 75% of 1 second or 750ms. A smaller number will make the spinner rotate faster while a larger number will make it slower.

3. The final local variable is called spinstr. This is a string for which each character is a frame in our spinner animation. As you can see, the string is 4 characters long, therefore we have four frames in our animation.

Moving along we get to the meat of the function, the primary loop.

while [ "$(ps a | awk '{print $1}' | grep -w $pid)" ]; do

The loop condition does three things:

1. ps a will show all processes.

2. awk '{print $1}' will extract the pid column of the process list.

3. grep -w $pid will look for the process ID of our background task in the list of PIDs printed by awk.

The loop is conditioned on the return value of grep, the last command in our pipe chain. If grep finds a match in our PID list it will return 0, otherwise it will return 1.

The next several lines do the hard work of displaying our animation. I’ve created two images to help illustrate what’s going on.

First, I remove the first character from the string and save the remaining characters into a temp.

local temp=${spinstr#?}

Then I use printf to output the first character of spinstr, which contains our animation. Only the first character is output because I use the %c format string.

printf " [%c]  " "$spinstr"

These two steps are illustrated below.

Finally, I shift spinstr by constructing a new string that contains the value of temp and all characters from spinstr that aren’t in temp.

local spinstr=$temp${spinstr%"$temp"}

The first part of this, the character rotation, appears as step 3 and the last part, the assignment to spinstr appears as step 4.

Using the Bash Spinner

When you have a task to run that will take a large (or unknown) amount of time invoke it in a background subshell like this:

(a_long_running_task) &

Then, immediately following that invocation, call the spinner and pass it the PID of the subshell you invoked.

spinner $!

The $! is a bash internal variable for the PID of the last job run in the background. In this case, it will give us the PID of the bash shell executing our long running task.

When it’s all said and done you’ll have a nice and simple Bash spinner like the one below.

Other posts you might like:

1. Introducing Bash Tips and Tricks
2. Showing File Download Progress Using Wget

Ran across this very informative video on using top. I use top like most people, which is to say I almost never touch any of the options. I have recently started to use htop quite a bit as well, since for me it has more sensible defaults and is more useful right out of the box. This video shows how useful and powerful top really is, and has motivated me spend a bit more time actually digging into the program, its options, and documentation.

Something else I learned from this video: using watch and /proc/interrupts to visualize interrupt load across CPUs. Very cool.

How important is your email account? How much damage can someone cause if they got access to your email? In my opinion, your email account is more important than any single other account you might have. The reason is simple: with access to your email an attacker can access every account you have. All they have to do is ask the target site, let’s say Facebook, to reset your password. Facebook will happily send an email to you with a link to change your password, allowing the attacker to take control of your Facebook profile. Ouch. Imagine the amount of damage someone can do if they had unfettered access to your email for 8 hours, approximately the amount of time we all sleep each night. 8 hours before you might notice anything is wrong. Double ouch.

Most people take security for granted. They use the same password everywhere. If they are security conscious they might use a different password for their bank accounts. The same password. Everywhere. That means anytime a web site is compromised not only do the attackers have your standard password but they also have your email address. Email address + password = a very simple way for an attacker to automatically scan for potential accounts to compromise. You don’t have to be targeted individually because the ability to automatically try every email and password in the compromised database is trivial to acquire.

Google’s new 2-step authentication is a god send. In security parlance it provides what is known as Two-factor Authentication. The basic premise is simple: by combining two secrets from two separate sources you can achieve better security. One source should be something you know–the password you remember, for example. The other secret should come from a source that is in your physical possession, often referred to as “something you have”. For Google’s 2-step authentication the first secret is your Google account password. The second secret comes from an application that is installed on your phone–something you have.

If you haven’t enabled it you should. Here’s why:

1. It uses your phone. You always have your phone on your person or very near by.
2. It instantly makes you a non-easy target. Most attackers will happily move along when they realize your password is more than just HappyFunr0s3z.
3. There are two backup options in case you lose your phone or it gets stolen.
4. It stops the proliferation of your primary email password to other accounts and applications: for example your instant messaging client.

Google has made this really easy. It’s great that Google has realized how important your email account really is and that it deserves the level of security typically reserved for other services. For example, many banks will provide you with a hardware token that serves the same purpose that Google’s 2-step authentication does. Unfortunately, most people don’t take advantage of their bank’s hardware token either.

Do this now: go read Google’s announcement about 2-step authentication and then enable 2-step authentication for your Google account. Don’t wait. Just do it. Right. Now.

Why are you still here?

This post is part of a series: Bash Tips and Tricks.

In this inaugural post for the Bash Tips and Tricks series I’ve decided to lead off with a tip that I hacked up as a result of a personal pet peeve of mine: superfluous output to stdout. When I write scripts that must perform some long running task like downloading a large file I want the user to know something useful is happening and about how long it will take. The easiest option is to just spam stdout with as much output as possible, but I really don’t like doing this. Too much output might tell the user something is happening, but it can also hide more useful information like errors and can be deceptive as to how much progress the long running task has made. I prefer to show nice, clean output whenever possible, and I let the various programs spew to log files in case an error occurs.

This tip demonstrates how you can download a file using wget and show a nice, simple progress meter. It gives the user of your script exactly what they want to know: how much has been downloaded and approximately how much longer do I have to wait. Directly below you’ll find the solution. For those that are curious how it comes together keep reading and I’ll dissect the various bits of the solution to help you understand how it works.

The meat of the tip is in the call to wget. It chains together wget, grep, sed, and awk to get the percentage complete from wget. Using it is even easier:

The above usage would generate output that is nice and simple, like this:

Downloading patch-2.6.37.gz:  6%

That is a lot better than the default wget output, which is 9 lines of crap and 1 line of useful information:

--2011-02-10 08:08:31-- http://www.kernel.org/pub/linux/kernel/v2.6/patch-2.6.37.gz
Resolving www.kernel.org... 204.152.191.37, 149.20.20.133
Connecting to www.kernel.org|204.152.191.37|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 15430256 (15M) [application/x-gzip]
Saving to: "patch-2.6.37.gz"


100%[==========================================>] 15,430,256  1.79M/s   in 10s

2011-02-10 08:08:42 (1.48 MB/s) - "patch-2.6.37.gz" saved [15430256/15430256]

You can see this tip and the code behind it in the GitHub repository for the Bash Tips and Tricks series.

Here’s how it all works.

First, let’s understand the call to wget:

The default wget output is about 10 lines of which only 1 is really useful, the progress bar line. Does the user really care to know that the HTTP request was sent or that wget is waiting on a response? No, not really. Thankfully, wget has a command line option that can help us out.

To get a simplified status display from wget I use the --progress=dot option. This will give us status output as follows:

--2011-02-10 08:08:31-- http://www.kernel.org/pub/linux/kernel/v2.6/patch-2.6.37.gz
Resolving www.kernel.org... 204.152.191.37, 149.20.20.133
Connecting to www.kernel.org|204.152.191.37|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 15430256 (15M) [application/x-gzip]
Saving to: "patch-2.6.37.gz"


0K .......... .......... .......... .......... ..........  0%  189K 80s
  50K .......... .......... .......... .......... ..........  0%  655K 51s
 150K .......... .......... .......... .......... ..........  1%  686K 40s
 200K .......... .......... .......... .......... ..........  1%  743K 36s
 300K .......... .......... .......... .......... ..........  2%  767K 29s
 350K .......... .......... .......... .......... ..........  2%  770K 28s
 [... etc ...]

Now that wget is giving us some more reasonable to work with we can use grep to get the information that is relevant to us. For this tip the only thing I’m interested in are lines with percent signs (%) in them. Here is our call to grep:

The ‘–line-buffered- flag is very important. Without it we won’t see much if any progress changes because grep will be buffering until EOF. Telling grep to buffer until end of line means it will read one line, match the pattern, and write any matched output on a line by line basis. We now have something like this:

   0K .......... .......... .......... .......... ..........  0%  189K 80s
  50K .......... .......... .......... .......... ..........  0%  655K 51s
 150K .......... .......... .......... .......... ..........  1%  686K 40s
 200K .......... .......... .......... .......... ..........  1%  743K 36s
 300K .......... .......... .......... .......... ..........  2%  767K 29s
 350K .......... .......... .......... .......... ..........  2%  770K 28s
 [... etc ...]

The next step is to get rid anything that might be superfluous or break our word oriented parsing we’re going to do with awk. Namely, ew need to get rid of all those dots. The big reason here is that the number of dots that appears is not consistent. It might be that near the end of the file we get a line with only 3 dots. This isn’t good since we’re going to use awk in the last step to extract the information we want. So, let’s nuke those dots.

This is pretty straightforward. We have a regular expression that matches a dot (.) and we replace it with nothing, nuking all dots that might appear on the input stream. The only real trick here is to use the ‘-u’ flag, which is short for ‘–unbuffered’. As you can imagine, the reasoning for this is the same as in the grep explanation above, we want output to show up on stdin as soon as possible, not at EOF. After applying the sed transoformation our input to the last program in the pipeline will look like this:

   0K 0%  189K 80s
  50K 0%  655K 51s
 150K 1%  686K 40s
 200K 1%  743K 36s
 300K 2%  767K 29s
 350K 2%  770K 28s

We are going to use awk to extract the information we want. Our invocation of awk is straightforward:

The fields are separated by space and we take the 2nd field with is the percent of the download that we have completed. The backspace characters (\b) are written first to back us up on the terminal then we write out a four character percentage completion string. awk is the last tool in our pipeline so it will now write to stdout, and this is what we’ll see:

  2%

A right aligned, space padded, four character string with the percentage of the download we’ve completed.

Other posts you might like:

1. Introducing Bash Tips and Tricks

This post is part of a series: Bash Tips and Tricks.

I’ve been hacking a lot of Bash lately. Tariten Trading uses Linux end-to-end so I’ve had ample opportunity to scrub off the rust that has accumulated on my shell scripting skills. I’ve decided to write an irregular series highlighting some of the more interesting Bash snippets I create and come across.

One of the great things about Bash is that it not only teaches you the syntax of the language, but it also imparts the syntax and philosophy of the GNU and Unix tool chain that underpins everything you do. Indeed, the syntax of the Bash is only a small part of truly learning to write effective Bash scripts; a much larger part involves learning the tools that underpin the entire environment, such as Sed and Awk.

This won’t be a tutorial on how to use Bash, but will instead focus on specific recipes that people familiar with the language and environment can put to use. I’ll try to keep each snippet short and sweet, and where available I’ll offer alternate solutions and source references such as links to relevant Stack Overflow questions.

All posts in the series, most recent is first:

1. Comparing Version Strings with Bash
2. Showing a Bash Spinner for Long Running Tasks
3. Showing File Download Progress Using Wget

The control room monkeys have been hard at work for the past several days bringing the site back online. Apparently I did some maintenance on the web server about a month ago and forgot to restart the service. Bit rot set in and various things got out of date and broken. Anyhow, this only came to my attention because of some recent events that I’ll go into more detail on in a later post. Needless to say, Fixed in the Next Release is now back online and functioning as expected. Expect more soon…

(photo credit: Urban~Spaceman on Flickr)

There was very long discussion about Fred Wilson’s post on the collateral damage that might be caused by Senator Dodd’s financial overhaul bill. In particular there is language in the bill modifying what an “accredited investor” is, significantly raising the financial bar for people to invest in startups as angel investors.

Folks are upset about this because it will make it harder for upstart businesses to get capital to start and grow. Suggested fixes include leaving the current language alone to lowering the bar based on invested capital to some artificially low number—Fred suggests investments up to $25k be exempted, for example.

I got into the discussion asking simply, “Why 25k?” The limit is entirely arbitrary so why set a limit at all. It is not the government’s responsibility to tell its citizens how to spend, invest, or waste their money. Setting these sorts of limits only confuses the legal situation and shifts the burden of responsibility in a fuzzy, non-deterministic way. The result is a mess of regulation that increases the cost of doing business with no real gain other than reducing bad publicity when some grandmother loses money investing in a young wiz-kid’s high-tech startup.

Proponents of investor accreditation suggest that the instruments are simply too complex for an average person to understand and it is too easy for sleaze-balls to take advantage of unsuspecting, uninformed people. This might be true, but then why does the government only provide protection around investments? The government does not tell me whether I’m suited to purchase a house nor do they advise for or against my trip to Las Vegas. This is simply selective nanny-state politics.

To be clear: the government absolutely has a role here. In my opinion that role is to ensure that sufficient law and regulation exist to ensure that people are provided sufficient information to make an informed decision—they should not be trying to define which of its citizens are wise enough to make such decisions. However, if practicality requires such an accredited investor status to exist then the government should solve the real problem. Accredited investors should be required to demonstrate proficiency to prove their ability to make informed risk decisions. Income and wealth levels are terrible predictors of this knowledge.

(photo credit: bluesquarething on Flickr)

Mark Suster, a two-time entrepreneur turned venture capitalist, is doing a series of posts on his blog about Entrepreneurial DNA. Mark’s series is going through 11 key attributes that successful entrepreneurs need to have. I have enjoyed reading the series thus far but I think Mark left out the most important attribute of all.

Mark’s list begins with what he believes to be the key attribute: tenacity. Quoting from the article:

Tenacity is probably the most important attribute in an entrepreneur.  It’s the person who never gives up – who never accepts “no” for an answer.  The world is filled with doubters who say that things can’t be done and then pronounce after the fact that they “knew it all along.”  Look at Google.  You think that anybody really believed 1999 that two young kids out of Stanford had a shot at unseating Yahoo!, Excite, Ask Jeeves and Lycos?

I agree with everything Mark is saying about not giving up, but I would disagree at its the “most important” attribute. In my mind the most critical requirement for any entrepreneur is whether they Get Things Done.

Let’s Git-R-Done

Larry the Cable Guy is an American Redneck comedian famous for the phrase, “Git-r-done”. In my mind, this should be the rallying cry of every founder as they work through obstacle after obstacle in their quest to start a company.

Getting this done is so key you might think of it as a prerequisite to starting a company. If you can’t get things done, it doesn’t matter how much tenacity you have. It doesn’t matter how hard you work. Nothing matters if you can’t move the ball forward.

In real world terms this can manifest itself in many ways. Ask yourself these questions:

• Am I working on the absolute most important task I could be?
• Am I working on something that will eliminate an obstacle to my success?
• Am I avoiding the hard choices that need to be made to evolve my company and product?
• Did I just walk away from the most important opportunity in the early life of my company because I thought it was impossible to grasp?
• When you see an obstacle in front of you is your initial reaction anything other than to move forward, finding a way through, around, or over the problem?

Seemingly every day or week or a young startup’s life the founders are presented with new and unknowable challenges. In fact, I tend of think of this as an indicator of potential success. If you don’t feel as if you’re on the razors edge then you’re not pushing hard enough.

Getting Things Done at Metreos

At Metreos there was one specific event that was critical to our success. It was a Monday morning, if I recall correctly, and we got a call from a potential customer. At this time, we had no customers at all, just a bunch of people kicking the tires. This customer had been evaluating the product but we couldn’t get them over the line in terms of buying. Their question to me was simple, “We’re about to buy a product from one of your competitors. We like yours better, but there is a key feature that is missing. If you can add that feature by Friday, we’ll go with yours instead.”

Now, calling what they wanted a feature is really doing it a disservice. What they really wanted was an entire application. You see, at Metreos we build development tools. This potential customer wanted to use our tools but they also needed this particular application more. So, the customer decided to see if they could get both: Metreos would very quickly build the application using our tools, and the customer would buy both the application and the development platform they wanted.

The only problem was the application they wanted could easily take 3 months to build and they had given us a week to prove the key tenet of our sales pitch, that using our development tools would significantly reduce the amount of time required to build these applications. Of course, in my mind, there was no way we weren’t going to get this done.

Late Friday afternoon we shipped the application to the customer. It certainly wasn’t polished, but it worked and it proved to them that not only did we really want their business but the underlying tenet of our product, that you could very quickly build telephony applications, was true. This was our first customer, and they went on to be our biggest and best customer as well.

Getting things done is the most important attribute any entrepreneur can have. If you can’t get things done then no amount of hard work will make you a success.

Other posts you might like:

1. Onwards and Upwards: Goodbye Cisco, Hello TBD