Merge pull request #71 from heroku/diet

Ship the Diet Buildpack!

CHANGELOG.md

-## v7 (2012-03-27)
-
-* remove build asset caching
-* prevent installation of devDependencies
-* add warning if no nodejs version specified

CONTRIBUTING.md

-## Testing buildpack changes using Anvil
-
-[Anvil](https://github.com/ddollar/anvil) is a generic build server for Heroku.
-
-```
-gem install anvil-cli
-```
-
-The [heroku-anvil CLI plugin](https://github.com/ddollar/heroku-anvil) is a wrapper for anvil.
-
-```
-heroku plugins:install https://github.com/ddollar/heroku-anvil
-```
-
-The [ddollar/test buildpack](https://github.com/ddollar/buildpack-test) is for testing things: it runs `bin/test` on your app.
-
-```
-heroku build -b ddollar/test # -b can also point to a local directory
-```
-
-## Compiling new versions of node and npm using Vulcan
-
-Install [vulcan](https://github.com/heroku/vulcan) and create your own build server. Use any
-app name you want and vulcan will remember it in a `~/.vulcan` config file.
-
-```
-gem install vulcan
-vulcan create builder-bob
-```
-
-Store your S3 credentials in `~/.aws/`
-
-```
-mkdir -p ~/.aws
-echo 'YOUR_AWS_KEY' > ~/.aws/key-nodejs.access
-echo 'YOUR_AWS_SECRET' > ~/.aws/key-nodejs.secret
-```
-
-Add a credentials exporter to your `.bash_profile` or `.bashrc`
-
-```
-setup_nodejs_env () {
-  export AWS_ID=$(cat ~/.aws/key-nodejs.access)
-  export AWS_SECRET=$(cat ~/.aws/key-nodejs.secret)
-  export S3_BUCKET="heroku-buildpack-nodejs"
-}
-```
-
-Build:
-
-```
-setup_nodejs_env
-support/package_nodejs <node-version>
-support/package_npm <npm-version>
-```
-
-## Publishing buildpack updates
-
-```
-heroku plugins:install https://github.com/heroku/heroku-buildpacks
-
-cd heroku-buildpack-nodejs
-git checkout master
-heroku buildpacks:publish heroku/nodejs
-```
-
-- Email [dos@heroku.com](mailto:dos@heroku.com) if changes are significant.
-- Add a [changelog item](https://devcenter.heroku.com/admin/changelog_items/new).
-- Update [Node Devcenter articles](https://devcenter.heroku.com/admin/articles/owned) as necessary.
-
-## Keeping up with the Nodeses
-
-- Run `npm info npm version` to find out the latest available version of npm.
-- Follow [@nodejs](https://twitter.com/nodejs) and [@npmjs](https://twitter.com/npmjs) on Twitter.
\ No newline at end of file

LICENSE

 MIT License:
 
-Copyright (C) 2012 Heroku, Inc.
+Copyright (C) 2012-2013 Heroku, Inc.
 
 Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
 

README.md

 Heroku Buildpack for Node.js
 ============================
 
-This is a [Heroku buildpack](http://devcenter.heroku.com/articles/buildpacks) for Node.js apps.
+This is the official [Heroku buildpack](http://devcenter.heroku.com/articles/buildpacks) for Node.js apps. If you fork this repository, please **update this README** to explain what your fork does and why it's special.
 
-The buildpack will detect your app as Node.js if it has a `package.json` file in the root.  It will use npm to install your dependencies, and vendors a version of the Node.js runtime into your slug.
 
-Example Usage
--------------
+How it Works
+------------
 
-    $ ls
-    Procfile  package.json  web.js
-
-    $ heroku create
-
-    $ git push heroku master
-    ...
-    -----> Heroku receiving push
-    -----> Fetching custom buildpack
-    -----> Node.js app detected
-    -----> Vendoring node 0.4.7
-    -----> Installing dependencies with npm 1.0.8
-           express@2.1.0 ./node_modules/express
-           ├── mime@1.2.2
-           ├── qs@0.3.1
-           └── connect@1.6.2
-           Dependencies installed
-
-Node.js and npm versions
-------------------------
-
-You can specify the versions of Node.js and npm your application requires using `package.json`
-
-```json
-{
-  "name": "myapp",
-  "version": "0.0.1",
-  "engines": {
-    "node": "~0.10.13",
-    "npm": "~1.3.2"
-  }
-}
-```
+Here's an overview of what this buildpack does:
 
-To list the available versions of Node.js and npm, see these manifests:
+- Uses the [semver.io](http://semver.io) webservice to find the latest version of node that satisfies the [engines.node semver range](https://npmjs.org/doc/json.html#engines) in your package.json.
+- Allows any recent version of node to be used, including [pre-release versions](http://semver.io/node.json).
+- Uses an [S3 caching proxy](https://github.com/heroku/s3pository#readme) of nodejs.org for faster downloads of the node binary.
+- Discourages use of dangerous semver ranges like `*` and `>0.10`.
+- Uses the version of `npm` that comes bundled with `node`.
+- Puts `node` and `npm` on the `PATH` so they can be executed with [heroku run](https://devcenter.heroku.com/articles/one-off-dynos#an-example-one-off-dyno).
+- Caches the `node_modules` directory across builds for fast deploys.
+- Doesn't use the cache if `node_modules` is checked into version control.
+- Runs `npm rebuild` if `node_modules` is checked into version control.
+- Always runs `npm install` to ensure [npm script hooks](https://npmjs.org/doc/misc/npm-scripts.html) are executed.
+- Always runs `npm prune` after restoring cached modules to ensure cleanup of unused dependencies.
+
+For more technical details, see the [heavily-commented compile script](https://github.com/heroku/heroku-buildpack-nodejs/blob/master/bin/compile).
 
-- [heroku-buildpack-nodejs.s3.amazonaws.com/manifest.nodejs](http://heroku-buildpack-nodejs.s3.amazonaws.com/manifest.nodejs)
-- [heroku-buildpack-nodejs.s3.amazonaws.com/manifest.npm](http://heroku-buildpack-nodejs.s3.amazonaws.com/manifest.npm)
 
 Documentation
 -------------
 
-For more information about buildpacks and Node.js, see these Dev Center articles:
+For more information about using Node.js and buildpacks on Heroku, see these Dev Center articles:
 
-- [Getting Started with Node.js on Heroku](https://devcenter.heroku.com/articles/nodejs)
 - [Heroku Node.js Support](https://devcenter.heroku.com/articles/nodejs-support)
+- [Getting Started with Node.js on Heroku](https://devcenter.heroku.com/articles/nodejs)
 - [Buildpacks](https://devcenter.heroku.com/articles/buildpacks)
 - [Buildpack API](https://devcenter.heroku.com/articles/buildpack-api)
 
+
+Legacy Compatibility
+--------------------
+
+For most Node.js apps this buildpack should work just fine. If, however, you're unable to deploy using this new version of the buildpack, you can get your app working again by using the legacy branch:
+
+```
+heroku config:set BUILDPACK_URL=https://github.com/heroku/heroku-buildpack-nodejs#legacy -a my-app
+git commit -am "empty" --allow-empty # force a git commit
+git push heroku master
+```
+
+Then please open a support ticket at [help.heroku.com](https://help.heroku.com/) so we can diagnose and get your app running on the default buildpack.
+
 Hacking
 -------
 
 To make changes to this buildpack, fork it on Github. Push up changes to your fork, then create a new Heroku app to test it, or configure an existing app to use your buildpack:
 
-```sh
+```
 # Create a new Heroku app that uses your buildpack
 heroku create --buildpack <your-github-url>
 
@@ -73,4 +64,26 @@ heroku config:set BUILDPACK_URL=<your-github-url>
 heroku config:set BUILDPACK_URL=<your-github-url>#your-branch
 ```
 
-For more detailed information about testing buildpacks, see [CONTRIBUTING.md](CONTRIBUTING.md)
\ No newline at end of file
+For more detailed information about testing buildpacks, see [CONTRIBUTING.md](CONTRIBUTING.md)
+
+
+Testing
+-------
+
+[Anvil](https://github.com/ddollar/anvil) is a generic build server for Heroku.
+
+```
+gem install anvil-cli
+```
+
+The [heroku-anvil CLI plugin](https://github.com/ddollar/heroku-anvil) is a wrapper for anvil.
+
+```
+heroku plugins:install https://github.com/ddollar/heroku-anvil
+```
+
+The [ddollar/test](https://github.com/ddollar/buildpack-test) buildpack runs `bin/test` on your app/buildpack.
+
+```
+heroku build -b ddollar/test # -b can also point to a local directory
+```

bin/common.sh

+error() {
+  echo " !     $*" >&2
+  exit 1
+}
+
+status() {
+  echo "-----> $*"
+}
+
+protip() {
+  echo
+  echo "PRO TIP: $*" | indent
+  echo "See https://devcenter.heroku.com/articles/nodejs-support" | indent
+  echo
+}
+
+# sed -l basically makes sed replace and buffer through stdin to stdout
+# so you get updates while the command runs and dont wait for the end
+# e.g. npm install | indent
+indent() {
+  c='s/^/       /'
+  case $(uname) in
+    Darwin) sed -l "$c";; # mac/bsd sed: -l buffers on line boundaries
+    *)      sed -u "$c";; # unix/gnu sed: -u unbuffered (arbitrary) chunks of data
+  esac
+}
+
+cat_npm_debug_log() {
+  test -f $build_dir/npm-debug.log && cat $build_dir/npm-debug.log
+}

bin/compile

 #!/usr/bin/env bash
-# bin/compile <build-dir> <cache-dir>
-
-# fail fast
-set -e
-
-# debug
-# set -x
-
-# clean up leaking environment
-unset GIT_DIR
-
-# config
-SCONS_VERSION="1.2.0"
-S3_BUCKET="heroku-buildpack-nodejs"
-
-# parse and derive params
-BUILD_DIR=$1
-CACHE_DIR=$2
-LP_DIR=`cd $(dirname $0); cd ..; pwd`
-
-function error() {
-  echo " !     $*" >&2
-  exit 1
-}
-
-function mktmpdir() {
-  dir=$(mktemp -t node-$1-XXXX)
-  rm -rf $dir
-  mkdir -p $dir
-  echo $dir
-}
-
-function indent() {
-  c='s/^/       /'
-  case $(uname) in
-    Darwin) sed -l "$c";;
-    *)      sed -u "$c";;
-  esac
-}
-
-function run_npm() {
-  command="$1"
-
-  cd $BUILD_DIR
-  HOME="$BUILD_DIR" $VENDORED_NODE/bin/node $VENDORED_NPM/cli.js $command 2>&1 | indent
-
-  if [ "${PIPESTATUS[*]}" != "0 0" ]; then
-    echo " !     Failed to $command dependencies with npm"
-    exit 1
-  fi
-}
-
-function manifest_versions() {
-  curl "http://${S3_BUCKET}.s3.amazonaws.com/manifest.${1}" -s -o - | tr -s '\n' ' '
-}
-
-function resolve_version() {
-  available_versions="$1"
-  requested_version="$2"
-  default_version="$3"
-
-  args=""
-  for version in $available_versions; do args="${args} -v \"${version}\""; done
-
-  if [ "$2" == "" ]; then
-    args="${args} -r \"${default_version}\"";
-  else
-    args="${args} -r \"${requested_version}\"";
-  fi
-
-  evaluated_versions=$(eval $bootstrap_node/bin/node $LP_DIR/vendor/node-semver/bin/semver ${args} || echo "")
-  echo "$evaluated_versions" | tail -n 1
-}
-
-function package_engine_version() {
-  version=$(cat $BUILD_DIR/package.json | $bootstrap_node/bin/node $LP_DIR/vendor/json/json engines.$1 2>/dev/null)
-  if [ $? == 0 ]; then
-    echo $version | sed -e 's/\([<>=]\) /\1/g'
-  fi
-}
-
-function package_resolve_version() {
-  engine="$1"
-  resolved_version=$(resolve_version "${engine_versions[$engine]}" "${engine_requests[$engine]}" "${engine_defaults[$engine]}")
-
-  if [ "${resolved_version}" == "" ]; then
-    error "Requested engine $engine version ${engine_requests[$engine]} does not match available versions: ${engine_versions[$engine]}"
-  else
-    echo $resolved_version
-  fi
-}
-
-function package_download() {
-  engine="$1"
-  version="$2"
-  location="$3"
-
-  mkdir -p $location
-  package="http://${S3_BUCKET}.s3.amazonaws.com/$engine-$version.tgz"
-  curl $package -s -o - | tar xzf - -C $location
-}
-
-function cat_npm_debug_log() {
-  if [ -f $BUILD_DIR/npm-debug.log ]; then
-    cat $BUILD_DIR/npm-debug.log
-  fi
-}
-
-trap cat_npm_debug_log EXIT
-
-bootstrap_node=$(mktmpdir bootstrap_node)
-package_download "nodejs" "0.4.7" $bootstrap_node
-
-# make some associative arrays
-declare -A engine_versions
-declare -A engine_defaults
-declare -A engine_requests
-
-engine_defaults["node"]="0.10.x"
-engine_defaults["npm"]="1.3.x"
-
-engine_versions["node"]=$(manifest_versions "nodejs")
-engine_requests["node"]=$(package_engine_version "node")
-
-engine_versions["npm"]=$(manifest_versions "npm")
-engine_requests["npm"]=$(package_engine_version "npm")
-
-echo "-----> Resolving engine versions"
-
-# add a warning if no version of node specified
-if [ "${engine_requests["node"]}" == "" ]; then
-  echo
-  echo "WARNING: No version of Node.js specified in package.json, see:" | indent
-  echo "https://devcenter.heroku.com/articles/nodejs-support#versions" | indent
-  echo
+
+set -e            # fail fast
+set -o pipefail   # don't ignore exit codes when piping output
+# set -x          # enable debugging
+
+# Configure directories
+build_dir=$1
+cache_dir=$2
+bp_dir=$(cd $(dirname $0); cd ..; pwd)
+
+# Load some convenience functions like status(), echo(), and indent()
+source $bp_dir/bin/common.sh
+
+# Output npm debug info on error
+trap cat_npm_debug_log ERR
+
+# Look in package.json's engines.node field for a semver range
+semver_range=$(cat $build_dir/package.json | $bp_dir/vendor/jq -r .engines.node)
+
+# Resolve node version using semver.io
+node_version=$(curl --silent --get --data-urlencode "range=${semver_range}" http://semver.io/node/resolve)
+
+# Recommend using semver ranges in a safe manner
+if [ "$semver_range" == "null" ]; then
+  protip "Specify a node version in package.json"
+  semver_range=""
+elif [ "$semver_range" == "*" ]; then
+  protip "Avoid using semver ranges like '*' in engines.node"
+elif [ ${semver_range:0:1} == ">" ]; then
+  protip "Avoid using semver ranges starting with '>' in engines.node"
+fi
+
+# Output info about requested range and resolved node version
+if [ "$semver_range" == "" ]; then
+  status "Defaulting to latest stable node: $node_version"
+else
+  status "Requested node range:  $semver_range"
+  status "Resolved node version: $node_version"
+fi
+
+# Download node from Heroku's S3 mirror of nodejs.org/dist
+status "Downloading and installing node"
+node_url="http://s3pository.heroku.com/node/v$node_version/node-v$node_version-linux-x64.tar.gz"
+curl $node_url -s -o - | tar xzf - -C $build_dir
+
+# Move node into ./vendor and make it executable
+mkdir -p $build_dir/vendor
+mv $build_dir/node-v$node_version-linux-x64 $build_dir/vendor/node
+chmod +x $build_dir/vendor/node/bin/*
+PATH=$PATH:$build_dir/vendor/node/bin
+
+# Run subsequent node/npm commands from the build path
+cd $build_dir
+
+# If node_modules directory is checked into source control then
+# rebuild any native deps. Otherwise, restore from the build cache.
+if test -d $build_dir/node_modules; then
+  status "Using existing node_modules directory"
+  status "Rebuilding any native dependencies"
+  npm rebuild 2>&1 | indent
+elif test -d $cache_dir/node_modules; then
+  status "Restoring node_modules from cache"
+  cp -r $cache_dir/node_modules $build_dir/
 fi
 
-NODE_VERSION=$(package_resolve_version "node")
-echo "Using Node.js version: ${NODE_VERSION}" | indent
-
-NPM_VERSION=$(package_resolve_version "npm")
-echo "Using npm version: ${NPM_VERSION}" | indent
-
-# cache directories
-CACHE_STORE_DIR="$CACHE_DIR/node_modules/$NODE_VERSION/$NPM_VERSION"
-CACHE_TARGET_DIR="$BUILD_DIR/node_modules"
-
-# s3 packages
-NODE_PACKAGE="http://${S3_BUCKET}.s3.amazonaws.com/nodejs-${NODE_VERSION}.tgz"
-NPM_PACKAGE="http://${S3_BUCKET}.s3.amazonaws.com/npm-${NPM_VERSION}.tgz"
-SCONS_PACKAGE="http://${S3_BUCKET}.s3.amazonaws.com/scons-${SCONS_VERSION}.tgz"
-
-# vendor directories
-VENDORED_NODE="$(mktmpdir node)"
-VENDORED_NPM="$(mktmpdir npm)"
-VENDORED_SCONS="$(mktmpdir scons)"
-
-# download and unpack packages
-echo "-----> Fetching Node.js binaries"
-package_download "nodejs" "${NODE_VERSION}" "${VENDORED_NODE}"
-package_download "npm" "${NPM_VERSION}" "${VENDORED_NPM}"
-package_download "scons" "${SCONS_VERSION}" "${VENDORED_SCONS}"
-
-# vendor node into the slug
-PATH="$BUILD_DIR/bin:$PATH"
-echo "-----> Vendoring node into slug"
-mkdir -p "$BUILD_DIR/bin"
-cp "$VENDORED_NODE/bin/node" "$BUILD_DIR/bin/node"
-
-# setting up paths for building
-PATH="$VENDORED_SCONS:$VENDORED_NODE/bin:$PATH"
-INCLUDE_PATH="$VENDORED_NODE/include"
-export CPATH="$INCLUDE_PATH:$CPATH"
-export CPPPATH="$INCLUDE_PATH:$CPPPATH"
-
-# install dependencies with npm
-echo "-----> Installing dependencies with npm"
-run_npm "install --production"
-run_npm "rebuild"
-echo "Dependencies installed" | indent
-
-echo "-----> Building runtime environment"
-mkdir -p $BUILD_DIR/.profile.d
-echo "export PATH=\"\$HOME/bin:\$HOME/node_modules/.bin:\$PATH\"" > $BUILD_DIR/.profile.d/nodejs.sh
+# Make npm output to STDOUT instead of its default STDERR
+status "Installing dependencies"
+npm install --production 2>&1 | indent
+
+status "Pruning unused dependencies"
+npm prune 2>&1 | indent
+
+status "Caching node_modules directory for future builds"
+rm -rf $cache_dir
+mkdir -p $cache_dir
+test -d $build_dir/node_modules && cp -r $build_dir/node_modules $cache_dir/
+
+status "Cleaning up node-gyp and npm artifacts"
+rm -rf "$build_dir/.node-gyp"
+rm -rf "$build_dir/.npm"
+
+# Update the PATH
+status "Building runtime environment"
+mkdir -p $build_dir/.profile.d
+echo "export PATH=\"\$HOME/vendor/node/bin:\$HOME/bin:\$HOME/node_modules/.bin:\$PATH\"" > $build_dir/.profile.d/nodejs.sh
+
+# Post package.json to nomnom service
+# Use a subshell so failures won't break the build.
+(
+  curl \
+    --data @$build_dir/package.json \
+    --fail \
+    --silent \
+    --request POST \
+    --header "content-type: application/json" \
+    https://nomnom.heroku.com/?request_id=$REQUEST_ID
+) &

bin/release

-#!/usr/bin/env bash
-# bin/release <build-dir>
-
-cat <<EOF
----
-config_vars:
-  PATH: bin:node_modules/.bin:/usr/local/bin:/usr/bin:/bin
-EOF

bin/test

 #!/usr/bin/env bash
-
-#
-# Create a Heroku app with the following buildpack:
-# https://github.com/ddollar/buildpack-test
-#
-# Push this Node.js buildpack to that Heroku app to
-# run the tests
-#
+# See CONTRIBUTING.md for info on running these tests.
 
 testDetectWithPackageJson() {
-  detect "package-json-version"
+  detect "stable-node"
   assertCaptured "Node.js"
   assertCapturedSuccess
 }
@@ -19,48 +12,113 @@ testDetectWithoutPackageJson() {
   assertCapturedError 1 ""
 }
 
-testPackageJsonWithVersion() {
-  compile "package-json-version"
-  assertNotCaptured "WARNING: No version of Node.js specified"
-  assertCaptured "Using Node.js version: 0.6.11"
-  assertCaptured "Using npm version: 1.1.9"
+testNoVersion() {
+  compile "no-version"
+  assertCaptured "PRO TIP: Specify a node version in package.json"
+  assertCaptured "Defaulting to latest stable node"
   assertCapturedSuccess
 }
 
-testPackageJsonWithoutVersion() {
-  compile "package-json-noversion"
-  assertCaptured "WARNING: No version of Node.js specified"
-  assertCaptured "Using Node.js version: 0.10"
-  assertCaptured "Using npm version: 1.3"
+testDangerousRangeStar() {
+  compile "dangerous-range-star"
+  assertCaptured "PRO TIP: Avoid using semver ranges like '*'"
+  assertCaptured "Requested node range:  *"
+  assertCaptured "Resolved node version: 0.10"
   assertCapturedSuccess
 }
 
-testPackageJsonWithInvalidVersion() {
-  compile "package-json-invalidversion"
-  assertCapturedError 1 "Requested engine npm version 1.1.5 does not"
+testDangerousRangeGreaterThan() {
+  compile "dangerous-range-greater-than"
+  assertCaptured "PRO TIP: Avoid using semver ranges starting with '>'"
+  assertCaptured "Requested node range:  >"
+  assertCaptured "Resolved node version: 0.10."
+  assertCapturedSuccess
 }
 
-testNothingCached() {
-  cache=$(mktmpdir)
-  compile "package-json-version" $cache
+testRangeWithSpace() {
+  compile "range-with-space"
+  assertCaptured "Requested node range:  >= 0.8.x"
+  assertCaptured "Resolved node version: 0.10."
+  assertCapturedSuccess
+}
+
+testStableVersion() {
+  compile "stable-node"
+  assertNotCaptured "PRO TIP: Avoid using semver"
+  assertNotCaptured "PRO TIP: Specify"
+  assertCaptured "Resolved node version"
+  assertCapturedSuccess
+}
+
+testUnstableVersion() {
+  compile "unstable-version"
+  assertCaptured "Requested node range:  >0.11.0"
+  assertCaptured "Resolved node version: 0.11."
   assertCapturedSuccess
-  assertEquals "0" "$(ls -1 $cache | wc -l)"
 }
 
 testProfileCreated() {
-  compile "package-json-version"
+  compile "stable-node"
   assertCaptured "Building runtime environment"
-  assertFile "export PATH=\"\$HOME/bin:\$HOME/node_modules/.bin:\$PATH\"" ".profile.d/nodejs.sh"