Commit 4843a79c authored by Zeke Sikelianos's avatar Zeke Sikelianos
Browse files

Merge pull request #71 from heroku/diet

Ship the Diet Buildpack!
parents c05e23bc 27641610
## v7 (2012-03-27)
* remove build asset caching
* prevent installation of devDependencies
* add warning if no nodejs version specified
## Testing buildpack changes using Anvil
[Anvil](https://github.com/ddollar/anvil) is a generic build server for Heroku.
```
gem install anvil-cli
```
The [heroku-anvil CLI plugin](https://github.com/ddollar/heroku-anvil) is a wrapper for anvil.
```
heroku plugins:install https://github.com/ddollar/heroku-anvil
```
The [ddollar/test buildpack](https://github.com/ddollar/buildpack-test) is for testing things: it runs `bin/test` on your app.
```
heroku build -b ddollar/test # -b can also point to a local directory
```
## Compiling new versions of node and npm using Vulcan
Install [vulcan](https://github.com/heroku/vulcan) and create your own build server. Use any
app name you want and vulcan will remember it in a `~/.vulcan` config file.
```
gem install vulcan
vulcan create builder-bob
```
Store your S3 credentials in `~/.aws/`
```
mkdir -p ~/.aws
echo 'YOUR_AWS_KEY' > ~/.aws/key-nodejs.access
echo 'YOUR_AWS_SECRET' > ~/.aws/key-nodejs.secret
```
Add a credentials exporter to your `.bash_profile` or `.bashrc`
```
setup_nodejs_env () {
export AWS_ID=$(cat ~/.aws/key-nodejs.access)
export AWS_SECRET=$(cat ~/.aws/key-nodejs.secret)
export S3_BUCKET="heroku-buildpack-nodejs"
}
```
Build:
```
setup_nodejs_env
support/package_nodejs <node-version>
support/package_npm <npm-version>
```
## Publishing buildpack updates
```
heroku plugins:install https://github.com/heroku/heroku-buildpacks
cd heroku-buildpack-nodejs
git checkout master
heroku buildpacks:publish heroku/nodejs
```
- Email [dos@heroku.com](mailto:dos@heroku.com) if changes are significant.
- Add a [changelog item](https://devcenter.heroku.com/admin/changelog_items/new).
- Update [Node Devcenter articles](https://devcenter.heroku.com/admin/articles/owned) as necessary.
## Keeping up with the Nodeses
- Run `npm info npm version` to find out the latest available version of npm.
- Follow [@nodejs](https://twitter.com/nodejs) and [@npmjs](https://twitter.com/npmjs) on Twitter.
\ No newline at end of file
MIT License: MIT License:
Copyright (C) 2012 Heroku, Inc. Copyright (C) 2012-2013 Heroku, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
......
Heroku Buildpack for Node.js Heroku Buildpack for Node.js
============================ ============================
This is a [Heroku buildpack](http://devcenter.heroku.com/articles/buildpacks) for Node.js apps. This is the official [Heroku buildpack](http://devcenter.heroku.com/articles/buildpacks) for Node.js apps. If you fork this repository, please **update this README** to explain what your fork does and why it's special.
The buildpack will detect your app as Node.js if it has a `package.json` file in the root. It will use npm to install your dependencies, and vendors a version of the Node.js runtime into your slug.
Example Usage How it Works
------------- ------------
$ ls Here's an overview of what this buildpack does:
Procfile package.json web.js
$ heroku create
$ git push heroku master
...
-----> Heroku receiving push
-----> Fetching custom buildpack
-----> Node.js app detected
-----> Vendoring node 0.4.7
-----> Installing dependencies with npm 1.0.8
express@2.1.0 ./node_modules/express
├── mime@1.2.2
├── qs@0.3.1
└── connect@1.6.2
Dependencies installed
Node.js and npm versions
------------------------
You can specify the versions of Node.js and npm your application requires using `package.json`
```json
{
"name": "myapp",
"version": "0.0.1",
"engines": {
"node": "~0.10.13",
"npm": "~1.3.2"
}
}
```
To list the available versions of Node.js and npm, see these manifests: - Uses the [semver.io](http://semver.io) webservice to find the latest version of node that satisfies the [engines.node semver range](https://npmjs.org/doc/json.html#engines) in your package.json.
- Allows any recent version of node to be used, including [pre-release versions](http://semver.io/node.json).
- Uses an [S3 caching proxy](https://github.com/heroku/s3pository#readme) of nodejs.org for faster downloads of the node binary.
- Discourages use of dangerous semver ranges like `*` and `>0.10`.
- Uses the version of `npm` that comes bundled with `node`.
- Puts `node` and `npm` on the `PATH` so they can be executed with [heroku run](https://devcenter.heroku.com/articles/one-off-dynos#an-example-one-off-dyno).
- Caches the `node_modules` directory across builds for fast deploys.
- Doesn't use the cache if `node_modules` is checked into version control.
- Runs `npm rebuild` if `node_modules` is checked into version control.
- Always runs `npm install` to ensure [npm script hooks](https://npmjs.org/doc/misc/npm-scripts.html) are executed.
- Always runs `npm prune` after restoring cached modules to ensure cleanup of unused dependencies.
For more technical details, see the [heavily-commented compile script](https://github.com/heroku/heroku-buildpack-nodejs/blob/master/bin/compile).
- [heroku-buildpack-nodejs.s3.amazonaws.com/manifest.nodejs](http://heroku-buildpack-nodejs.s3.amazonaws.com/manifest.nodejs)
- [heroku-buildpack-nodejs.s3.amazonaws.com/manifest.npm](http://heroku-buildpack-nodejs.s3.amazonaws.com/manifest.npm)
Documentation Documentation
------------- -------------
For more information about buildpacks and Node.js, see these Dev Center articles: For more information about using Node.js and buildpacks on Heroku, see these Dev Center articles:
- [Getting Started with Node.js on Heroku](https://devcenter.heroku.com/articles/nodejs)
- [Heroku Node.js Support](https://devcenter.heroku.com/articles/nodejs-support) - [Heroku Node.js Support](https://devcenter.heroku.com/articles/nodejs-support)
- [Getting Started with Node.js on Heroku](https://devcenter.heroku.com/articles/nodejs)
- [Buildpacks](https://devcenter.heroku.com/articles/buildpacks) - [Buildpacks](https://devcenter.heroku.com/articles/buildpacks)
- [Buildpack API](https://devcenter.heroku.com/articles/buildpack-api) - [Buildpack API](https://devcenter.heroku.com/articles/buildpack-api)
Legacy Compatibility
--------------------
For most Node.js apps this buildpack should work just fine. If, however, you're unable to deploy using this new version of the buildpack, you can get your app working again by using the legacy branch:
```
heroku config:set BUILDPACK_URL=https://github.com/heroku/heroku-buildpack-nodejs#legacy -a my-app
git commit -am "empty" --allow-empty # force a git commit
git push heroku master
```
Then please open a support ticket at [help.heroku.com](https://help.heroku.com/) so we can diagnose and get your app running on the default buildpack.
Hacking Hacking
------- -------
To make changes to this buildpack, fork it on Github. Push up changes to your fork, then create a new Heroku app to test it, or configure an existing app to use your buildpack: To make changes to this buildpack, fork it on Github. Push up changes to your fork, then create a new Heroku app to test it, or configure an existing app to use your buildpack:
```sh ```
# Create a new Heroku app that uses your buildpack # Create a new Heroku app that uses your buildpack
heroku create --buildpack <your-github-url> heroku create --buildpack <your-github-url>
...@@ -73,4 +64,26 @@ heroku config:set BUILDPACK_URL=<your-github-url> ...@@ -73,4 +64,26 @@ heroku config:set BUILDPACK_URL=<your-github-url>
heroku config:set BUILDPACK_URL=<your-github-url>#your-branch heroku config:set BUILDPACK_URL=<your-github-url>#your-branch
``` ```
For more detailed information about testing buildpacks, see [CONTRIBUTING.md](CONTRIBUTING.md) For more detailed information about testing buildpacks, see [CONTRIBUTING.md](CONTRIBUTING.md)
\ No newline at end of file
Testing
-------
[Anvil](https://github.com/ddollar/anvil) is a generic build server for Heroku.
```
gem install anvil-cli
```
The [heroku-anvil CLI plugin](https://github.com/ddollar/heroku-anvil) is a wrapper for anvil.
```
heroku plugins:install https://github.com/ddollar/heroku-anvil
```
The [ddollar/test](https://github.com/ddollar/buildpack-test) buildpack runs `bin/test` on your app/buildpack.
```
heroku build -b ddollar/test # -b can also point to a local directory
```
error() {
echo " ! $*" >&2
exit 1
}
status() {
echo "-----> $*"
}
protip() {
echo
echo "PRO TIP: $*" | indent
echo "See https://devcenter.heroku.com/articles/nodejs-support" | indent
echo
}
# sed -l basically makes sed replace and buffer through stdin to stdout
# so you get updates while the command runs and dont wait for the end
# e.g. npm install | indent
indent() {
c='s/^/ /'
case $(uname) in
Darwin) sed -l "$c";; # mac/bsd sed: -l buffers on line boundaries
*) sed -u "$c";; # unix/gnu sed: -u unbuffered (arbitrary) chunks of data
esac
}
cat_npm_debug_log() {
test -f $build_dir/npm-debug.log && cat $build_dir/npm-debug.log
}
#!/usr/bin/env bash #!/usr/bin/env bash
# bin/compile <build-dir> <cache-dir>
set -e # fail fast
# fail fast set -o pipefail # don't ignore exit codes when piping output
set -e # set -x # enable debugging
# debug # Configure directories
# set -x build_dir=$1
cache_dir=$2
# clean up leaking environment bp_dir=$(cd $(dirname $0); cd ..; pwd)
unset GIT_DIR
# Load some convenience functions like status(), echo(), and indent()
# config source $bp_dir/bin/common.sh
SCONS_VERSION="1.2.0"
S3_BUCKET="heroku-buildpack-nodejs" # Output npm debug info on error
trap cat_npm_debug_log ERR
# parse and derive params
BUILD_DIR=$1 # Look in package.json's engines.node field for a semver range
CACHE_DIR=$2 semver_range=$(cat $build_dir/package.json | $bp_dir/vendor/jq -r .engines.node)
LP_DIR=`cd $(dirname $0); cd ..; pwd`
# Resolve node version using semver.io
function error() { node_version=$(curl --silent --get --data-urlencode "range=${semver_range}" http://semver.io/node/resolve)
echo " ! $*" >&2
exit 1 # Recommend using semver ranges in a safe manner
} if [ "$semver_range" == "null" ]; then
protip "Specify a node version in package.json"
function mktmpdir() { semver_range=""
dir=$(mktemp -t node-$1-XXXX) elif [ "$semver_range" == "*" ]; then
rm -rf $dir protip "Avoid using semver ranges like '*' in engines.node"
mkdir -p $dir elif [ ${semver_range:0:1} == ">" ]; then
echo $dir protip "Avoid using semver ranges starting with '>' in engines.node"
} fi
function indent() { # Output info about requested range and resolved node version
c='s/^/ /' if [ "$semver_range" == "" ]; then
case $(uname) in status "Defaulting to latest stable node: $node_version"
Darwin) sed -l "$c";; else
*) sed -u "$c";; status "Requested node range: $semver_range"
esac status "Resolved node version: $node_version"
} fi
function run_npm() { # Download node from Heroku's S3 mirror of nodejs.org/dist
command="$1" status "Downloading and installing node"
node_url="http://s3pository.heroku.com/node/v$node_version/node-v$node_version-linux-x64.tar.gz"
cd $BUILD_DIR curl $node_url -s -o - | tar xzf - -C $build_dir
HOME="$BUILD_DIR" $VENDORED_NODE/bin/node $VENDORED_NPM/cli.js $command 2>&1 | indent
# Move node into ./vendor and make it executable
if [ "${PIPESTATUS[*]}" != "0 0" ]; then mkdir -p $build_dir/vendor
echo " ! Failed to $command dependencies with npm" mv $build_dir/node-v$node_version-linux-x64 $build_dir/vendor/node
exit 1 chmod +x $build_dir/vendor/node/bin/*
fi PATH=$PATH:$build_dir/vendor/node/bin
}
# Run subsequent node/npm commands from the build path
function manifest_versions() { cd $build_dir
curl "http://${S3_BUCKET}.s3.amazonaws.com/manifest.${1}" -s -o - | tr -s '\n' ' '
} # If node_modules directory is checked into source control then
# rebuild any native deps. Otherwise, restore from the build cache.
function resolve_version() { if test -d $build_dir/node_modules; then
available_versions="$1" status "Using existing node_modules directory"
requested_version="$2" status "Rebuilding any native dependencies"
default_version="$3" npm rebuild 2>&1 | indent
elif test -d $cache_dir/node_modules; then
args="" status "Restoring node_modules from cache"
for version in $available_versions; do args="${args} -v \"${version}\""; done cp -r $cache_dir/node_modules $build_dir/
if [ "$2" == "" ]; then
args="${args} -r \"${default_version}\"";
else
args="${args} -r \"${requested_version}\"";
fi
evaluated_versions=$(eval $bootstrap_node/bin/node $LP_DIR/vendor/node-semver/bin/semver ${args} || echo "")
echo "$evaluated_versions" | tail -n 1
}
function package_engine_version() {
version=$(cat $BUILD_DIR/package.json | $bootstrap_node/bin/node $LP_DIR/vendor/json/json engines.$1 2>/dev/null)
if [ $? == 0 ]; then
echo $version | sed -e 's/\([<>=]\) /\1/g'
fi
}
function package_resolve_version() {
engine="$1"
resolved_version=$(resolve_version "${engine_versions[$engine]}" "${engine_requests[$engine]}" "${engine_defaults[$engine]}")
if [ "${resolved_version}" == "" ]; then
error "Requested engine $engine version ${engine_requests[$engine]} does not match available versions: ${engine_versions[$engine]}"
else
echo $resolved_version
fi
}
function package_download() {
engine="$1"
version="$2"
location="$3"
mkdir -p $location
package="http://${S3_BUCKET}.s3.amazonaws.com/$engine-$version.tgz"
curl $package -s -o - | tar xzf - -C $location
}
function cat_npm_debug_log() {
if [ -f $BUILD_DIR/npm-debug.log ]; then
cat $BUILD_DIR/npm-debug.log
fi
}
trap cat_npm_debug_log EXIT
bootstrap_node=$(mktmpdir bootstrap_node)
package_download "nodejs" "0.4.7" $bootstrap_node
# make some associative arrays
declare -A engine_versions
declare -A engine_defaults
declare -A engine_requests
engine_defaults["node"]="0.10.x"
engine_defaults["npm"]="1.3.x"
engine_versions["node"]=$(manifest_versions "nodejs")
engine_requests["node"]=$(package_engine_version "node")
engine_versions["npm"]=$(manifest_versions "npm")
engine_requests["npm"]=$(package_engine_version "npm")
echo "-----> Resolving engine versions"
# add a warning if no version of node specified
if [ "${engine_requests["node"]}" == "" ]; then
echo
echo "WARNING: No version of Node.js specified in package.json, see:" | indent
echo "https://devcenter.heroku.com/articles/nodejs-support#versions" | indent
echo
fi fi
NODE_VERSION=$(package_resolve_version "node") # Make npm output to STDOUT instead of its default STDERR
echo "Using Node.js version: ${NODE_VERSION}" | indent status "Installing dependencies"
npm install --production 2>&1 | indent
NPM_VERSION=$(package_resolve_version "npm")
echo "Using npm version: ${NPM_VERSION}" | indent status "Pruning unused dependencies"
npm prune 2>&1 | indent
# cache directories
CACHE_STORE_DIR="$CACHE_DIR/node_modules/$NODE_VERSION/$NPM_VERSION" status "Caching node_modules directory for future builds"
CACHE_TARGET_DIR="$BUILD_DIR/node_modules" rm -rf $cache_dir
mkdir -p $cache_dir
# s3 packages test -d $build_dir/node_modules && cp -r $build_dir/node_modules $cache_dir/
NODE_PACKAGE="http://${S3_BUCKET}.s3.amazonaws.com/nodejs-${NODE_VERSION}.tgz"
NPM_PACKAGE="http://${S3_BUCKET}.s3.amazonaws.com/npm-${NPM_VERSION}.tgz" status "Cleaning up node-gyp and npm artifacts"
SCONS_PACKAGE="http://${S3_BUCKET}.s3.amazonaws.com/scons-${SCONS_VERSION}.tgz" rm -rf "$build_dir/.node-gyp"
rm -rf "$build_dir/.npm"
# vendor directories
VENDORED_NODE="$(mktmpdir node)" # Update the PATH
VENDORED_NPM="$(mktmpdir npm)" status "Building runtime environment"
VENDORED_SCONS="$(mktmpdir scons)" mkdir -p $build_dir/.profile.d
echo "export PATH=\"\$HOME/vendor/node/bin:\$HOME/bin:\$HOME/node_modules/.bin:\$PATH\"" > $build_dir/.profile.d/nodejs.sh
# download and unpack packages
echo "-----> Fetching Node.js binaries" # Post package.json to nomnom service
package_download "nodejs" "${NODE_VERSION}" "${VENDORED_NODE}" # Use a subshell so failures won't break the build.
package_download "npm" "${NPM_VERSION}" "${VENDORED_NPM}" (
package_download "scons" "${SCONS_VERSION}" "${VENDORED_SCONS}" curl \
--data @$build_dir/package.json \
# vendor node into the slug --fail \
PATH="$BUILD_DIR/bin:$PATH" --silent \
echo "-----> Vendoring node into slug" --request POST \
mkdir -p "$BUILD_DIR/bin" --header "content-type: application/json" \
cp "$VENDORED_NODE/bin/node" "$BUILD_DIR/bin/node" https://nomnom.heroku.com/?request_id=$REQUEST_ID
) &
# setting up paths for building
PATH="$VENDORED_SCONS:$VENDORED_NODE/bin:$PATH"
INCLUDE_PATH="$VENDORED_NODE/include"
export CPATH="$INCLUDE_PATH:$CPATH"
export CPPPATH="$INCLUDE_PATH:$CPPPATH"
# install dependencies with npm
echo "-----> Installing dependencies with npm"
run_npm "install --production"
run_npm "rebuild"
echo "Dependencies installed" | indent
echo "-----> Building runtime environment"
mkdir -p $BUILD_DIR/.profile.d
echo "export PATH=\"\$HOME/bin:\$HOME/node_modules/.bin:\$PATH\"" > $BUILD_DIR/.profile.d/nodejs.sh
#!/usr/bin/env bash
# bin/release <build-dir>
cat <<EOF
---
config_vars:
PATH: bin:node_modules/.bin:/usr/local/bin:/usr/bin:/bin
EOF
#!/usr/bin/env bash #!/usr/bin/env bash
# See CONTRIBUTING.md for info on running these tests.
#
# Create a Heroku app with the following buildpack:
# https://github.com/ddollar/buildpack-test
#
# Push this Node.js buildpack to that Heroku app to
# run the tests
#
testDetectWithPackageJson() { testDetectWithPackageJson() {
detect "package-json-version" detect "stable-node"
assertCaptured "Node.js" assertCaptured "Node.js"
assertCapturedSuccess assertCapturedSuccess
} }
...@@ -19,48 +12,113 @@ testDetectWithoutPackageJson() { ...@@ -19,48 +12,113 @@ testDetectWithoutPackageJson() {
assertCapturedError 1 "" assertCapturedError 1 ""
} }
testPackageJsonWithVersion() { testNoVersion() {
compile "package-json-version" compile "no-version"
assertNotCaptured "WARNING: No version of Node.js specified" assertCaptured "PRO TIP: Specify a node version in package.json"
assertCaptured "Using Node.js version: 0.6.11" assertCaptured "Defaulting to latest stable node"
assertCaptured "Using npm version: 1.1.9"
assertCapturedSuccess assertCapturedSuccess
} }
testPackageJsonWithoutVersion() { testDangerousRangeStar() {
compile "package-json-noversion" compile "dangerous-range-star"
assertCaptured "WARNING: No version of Node.js specified" assertCaptured "PRO TIP: Avoid using semver ranges like '*'"
assertCaptured "Using Node.js version: 0.10" assertCaptured "Requested node range: *"
assertCaptured "Using npm version: 1.3" assertCaptured "Resolved node version: 0.10"
assertCapturedSuccess assertCapturedSuccess
} }
testPackageJsonWithInvalidVersion() { testDangerousRangeGreaterThan() {
compile "package-json-invalidversion" compile "dangerous-range-greater-than"
assertCapturedError 1 "Requested engine npm version 1.1.5 does not" assertCaptured "PRO TIP: Avoid using semver ranges starting with '>'"
assertCaptured "Requested node range: >"
assertCaptured "Resolved node version: 0.10."
assertCapturedSuccess
} }
testNothingCached() { testRangeWithSpace() {
cache=$(mktmpdir) compile "range-with-space"
compile "package-json-version" $cache assertCaptured "Requested node range: >= 0.8.x"
assertCaptured "Resolved node version: 0.10."
assertCapturedSuccess
}
testStableVersion() {
compile "stable-node"
assertNotCaptured "PRO TIP: Avoid using semver"
assertNotCaptured "PRO TIP: Specify"
assertCaptured "Resolved node version"
assertCapturedSuccess
}