Commit 4843a79c authored by Zeke Sikelianos's avatar Zeke Sikelianos
Browse files

Merge pull request #71 from heroku/diet

Ship the Diet Buildpack!
parents c05e23bc 27641610
## v7 (2012-03-27)
* remove build asset caching
* prevent installation of devDependencies
* add warning if no nodejs version specified
## Testing buildpack changes using Anvil
[Anvil](https://github.com/ddollar/anvil) is a generic build server for Heroku.
```
gem install anvil-cli
```
The [heroku-anvil CLI plugin](https://github.com/ddollar/heroku-anvil) is a wrapper for anvil.
```
heroku plugins:install https://github.com/ddollar/heroku-anvil
```
The [ddollar/test buildpack](https://github.com/ddollar/buildpack-test) is for testing things: it runs `bin/test` on your app.
```
heroku build -b ddollar/test # -b can also point to a local directory
```
## Compiling new versions of node and npm using Vulcan
Install [vulcan](https://github.com/heroku/vulcan) and create your own build server. Use any
app name you want and vulcan will remember it in a `~/.vulcan` config file.
```
gem install vulcan
vulcan create builder-bob
```
Store your S3 credentials in `~/.aws/`
```
mkdir -p ~/.aws
echo 'YOUR_AWS_KEY' > ~/.aws/key-nodejs.access
echo 'YOUR_AWS_SECRET' > ~/.aws/key-nodejs.secret
```
Add a credentials exporter to your `.bash_profile` or `.bashrc`
```
setup_nodejs_env () {
export AWS_ID=$(cat ~/.aws/key-nodejs.access)
export AWS_SECRET=$(cat ~/.aws/key-nodejs.secret)
export S3_BUCKET="heroku-buildpack-nodejs"
}
```
Build:
```
setup_nodejs_env
support/package_nodejs <node-version>
support/package_npm <npm-version>
```
## Publishing buildpack updates
```
heroku plugins:install https://github.com/heroku/heroku-buildpacks
cd heroku-buildpack-nodejs
git checkout master
heroku buildpacks:publish heroku/nodejs
```
- Email [dos@heroku.com](mailto:dos@heroku.com) if changes are significant.
- Add a [changelog item](https://devcenter.heroku.com/admin/changelog_items/new).
- Update [Node Devcenter articles](https://devcenter.heroku.com/admin/articles/owned) as necessary.
## Keeping up with the Nodeses
- Run `npm info npm version` to find out the latest available version of npm.
- Follow [@nodejs](https://twitter.com/nodejs) and [@npmjs](https://twitter.com/npmjs) on Twitter.
\ No newline at end of file
MIT License:
Copyright (C) 2012 Heroku, Inc.
Copyright (C) 2012-2013 Heroku, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
......
Heroku Buildpack for Node.js
============================
This is a [Heroku buildpack](http://devcenter.heroku.com/articles/buildpacks) for Node.js apps.
This is the official [Heroku buildpack](http://devcenter.heroku.com/articles/buildpacks) for Node.js apps. If you fork this repository, please **update this README** to explain what your fork does and why it's special.
The buildpack will detect your app as Node.js if it has a `package.json` file in the root. It will use npm to install your dependencies, and vendors a version of the Node.js runtime into your slug.
Example Usage
-------------
How it Works
------------
$ ls
Procfile package.json web.js
$ heroku create
$ git push heroku master
...
-----> Heroku receiving push
-----> Fetching custom buildpack
-----> Node.js app detected
-----> Vendoring node 0.4.7
-----> Installing dependencies with npm 1.0.8
express@2.1.0 ./node_modules/express
├── mime@1.2.2
├── qs@0.3.1
└── connect@1.6.2
Dependencies installed
Node.js and npm versions
------------------------
You can specify the versions of Node.js and npm your application requires using `package.json`
```json
{
"name": "myapp",
"version": "0.0.1",
"engines": {
"node": "~0.10.13",
"npm": "~1.3.2"
}
}
```
Here's an overview of what this buildpack does:
To list the available versions of Node.js and npm, see these manifests:
- Uses the [semver.io](http://semver.io) webservice to find the latest version of node that satisfies the [engines.node semver range](https://npmjs.org/doc/json.html#engines) in your package.json.
- Allows any recent version of node to be used, including [pre-release versions](http://semver.io/node.json).
- Uses an [S3 caching proxy](https://github.com/heroku/s3pository#readme) of nodejs.org for faster downloads of the node binary.
- Discourages use of dangerous semver ranges like `*` and `>0.10`.
- Uses the version of `npm` that comes bundled with `node`.
- Puts `node` and `npm` on the `PATH` so they can be executed with [heroku run](https://devcenter.heroku.com/articles/one-off-dynos#an-example-one-off-dyno).
- Caches the `node_modules` directory across builds for fast deploys.
- Doesn't use the cache if `node_modules` is checked into version control.
- Runs `npm rebuild` if `node_modules` is checked into version control.
- Always runs `npm install` to ensure [npm script hooks](https://npmjs.org/doc/misc/npm-scripts.html) are executed.
- Always runs `npm prune` after restoring cached modules to ensure cleanup of unused dependencies.
For more technical details, see the [heavily-commented compile script](https://github.com/heroku/heroku-buildpack-nodejs/blob/master/bin/compile).
- [heroku-buildpack-nodejs.s3.amazonaws.com/manifest.nodejs](http://heroku-buildpack-nodejs.s3.amazonaws.com/manifest.nodejs)
- [heroku-buildpack-nodejs.s3.amazonaws.com/manifest.npm](http://heroku-buildpack-nodejs.s3.amazonaws.com/manifest.npm)
Documentation
-------------
For more information about buildpacks and Node.js, see these Dev Center articles:
For more information about using Node.js and buildpacks on Heroku, see these Dev Center articles:
- [Getting Started with Node.js on Heroku](https://devcenter.heroku.com/articles/nodejs)
- [Heroku Node.js Support](https://devcenter.heroku.com/articles/nodejs-support)
- [Getting Started with Node.js on Heroku](https://devcenter.heroku.com/articles/nodejs)
- [Buildpacks](https://devcenter.heroku.com/articles/buildpacks)
- [Buildpack API](https://devcenter.heroku.com/articles/buildpack-api)
Legacy Compatibility
--------------------
For most Node.js apps this buildpack should work just fine. If, however, you're unable to deploy using this new version of the buildpack, you can get your app working again by using the legacy branch:
```
heroku config:set BUILDPACK_URL=https://github.com/heroku/heroku-buildpack-nodejs#legacy -a my-app
git commit -am "empty" --allow-empty # force a git commit
git push heroku master
```
Then please open a support ticket at [help.heroku.com](https://help.heroku.com/) so we can diagnose and get your app running on the default buildpack.
Hacking
-------
To make changes to this buildpack, fork it on Github. Push up changes to your fork, then create a new Heroku app to test it, or configure an existing app to use your buildpack:
```sh
```
# Create a new Heroku app that uses your buildpack
heroku create --buildpack <your-github-url>
......@@ -74,3 +65,25 @@ heroku config:set BUILDPACK_URL=<your-github-url>#your-branch
```
For more detailed information about testing buildpacks, see [CONTRIBUTING.md](CONTRIBUTING.md)
Testing
-------
[Anvil](https://github.com/ddollar/anvil) is a generic build server for Heroku.
```
gem install anvil-cli
```
The [heroku-anvil CLI plugin](https://github.com/ddollar/heroku-anvil) is a wrapper for anvil.
```
heroku plugins:install https://github.com/ddollar/heroku-anvil
```
The [ddollar/test](https://github.com/ddollar/buildpack-test) buildpack runs `bin/test` on your app/buildpack.
```
heroku build -b ddollar/test # -b can also point to a local directory
```
error() {
echo " ! $*" >&2
exit 1
}
status() {
echo "-----> $*"
}
protip() {
echo
echo "PRO TIP: $*" | indent
echo "See https://devcenter.heroku.com/articles/nodejs-support" | indent
echo
}
# sed -l basically makes sed replace and buffer through stdin to stdout
# so you get updates while the command runs and dont wait for the end
# e.g. npm install | indent
indent() {
c='s/^/ /'
case $(uname) in
Darwin) sed -l "$c";; # mac/bsd sed: -l buffers on line boundaries
*) sed -u "$c";; # unix/gnu sed: -u unbuffered (arbitrary) chunks of data
esac
}
cat_npm_debug_log() {
test -f $build_dir/npm-debug.log && cat $build_dir/npm-debug.log
}
#!/usr/bin/env bash
# bin/compile <build-dir> <cache-dir>
# fail fast
set -e
# debug
# set -x
# clean up leaking environment
unset GIT_DIR
# config
SCONS_VERSION="1.2.0"
S3_BUCKET="heroku-buildpack-nodejs"
# parse and derive params
BUILD_DIR=$1
CACHE_DIR=$2
LP_DIR=`cd $(dirname $0); cd ..; pwd`
function error() {
echo " ! $*" >&2
exit 1
}
function mktmpdir() {
dir=$(mktemp -t node-$1-XXXX)
rm -rf $dir
mkdir -p $dir
echo $dir
}
function indent() {
c='s/^/ /'
case $(uname) in
Darwin) sed -l "$c";;
*) sed -u "$c";;
esac
}
function run_npm() {
command="$1"
cd $BUILD_DIR
HOME="$BUILD_DIR" $VENDORED_NODE/bin/node $VENDORED_NPM/cli.js $command 2>&1 | indent
if [ "${PIPESTATUS[*]}" != "0 0" ]; then
echo " ! Failed to $command dependencies with npm"
exit 1
fi
}
function manifest_versions() {
curl "http://${S3_BUCKET}.s3.amazonaws.com/manifest.${1}" -s -o - | tr -s '\n' ' '
}
function resolve_version() {
available_versions="$1"
requested_version="$2"
default_version="$3"
args=""
for version in $available_versions; do args="${args} -v \"${version}\""; done
if [ "$2" == "" ]; then
args="${args} -r \"${default_version}\"";
else
args="${args} -r \"${requested_version}\"";
fi
evaluated_versions=$(eval $bootstrap_node/bin/node $LP_DIR/vendor/node-semver/bin/semver ${args} || echo "")
echo "$evaluated_versions" | tail -n 1
}
function package_engine_version() {
version=$(cat $BUILD_DIR/package.json | $bootstrap_node/bin/node $LP_DIR/vendor/json/json engines.$1 2>/dev/null)
if [ $? == 0 ]; then
echo $version | sed -e 's/\([<>=]\) /\1/g'
fi
}
function package_resolve_version() {
engine="$1"
resolved_version=$(resolve_version "${engine_versions[$engine]}" "${engine_requests[$engine]}" "${engine_defaults[$engine]}")
if [ "${resolved_version}" == "" ]; then
error "Requested engine $engine version ${engine_requests[$engine]} does not match available versions: ${engine_versions[$engine]}"
else
echo $resolved_version
fi
}
function package_download() {
engine="$1"
version="$2"
location="$3"
mkdir -p $location
package="http://${S3_BUCKET}.s3.amazonaws.com/$engine-$version.tgz"
curl $package -s -o - | tar xzf - -C $location
}
function cat_npm_debug_log() {
if [ -f $BUILD_DIR/npm-debug.log ]; then
cat $BUILD_DIR/npm-debug.log
fi
}
trap cat_npm_debug_log EXIT
bootstrap_node=$(mktmpdir bootstrap_node)
package_download "nodejs" "0.4.7" $bootstrap_node
# make some associative arrays
declare -A engine_versions
declare -A engine_defaults
declare -A engine_requests
engine_defaults["node"]="0.10.x"
engine_defaults["npm"]="1.3.x"
engine_versions["node"]=$(manifest_versions "nodejs")
engine_requests["node"]=$(package_engine_version "node")
engine_versions["npm"]=$(manifest_versions "npm")
engine_requests["npm"]=$(package_engine_version "npm")
echo "-----> Resolving engine versions"
# add a warning if no version of node specified
if [ "${engine_requests["node"]}" == "" ]; then
echo
echo "WARNING: No version of Node.js specified in package.json, see:" | indent
echo "https://devcenter.heroku.com/articles/nodejs-support#versions" | indent
echo
set -e # fail fast
set -o pipefail # don't ignore exit codes when piping output
# set -x # enable debugging
# Configure directories
build_dir=$1
cache_dir=$2
bp_dir=$(cd $(dirname $0); cd ..; pwd)
# Load some convenience functions like status(), echo(), and indent()
source $bp_dir/bin/common.sh
# Output npm debug info on error
trap cat_npm_debug_log ERR
# Look in package.json's engines.node field for a semver range
semver_range=$(cat $build_dir/package.json | $bp_dir/vendor/jq -r .engines.node)
# Resolve node version using semver.io
node_version=$(curl --silent --get --data-urlencode "range=${semver_range}" http://semver.io/node/resolve)
# Recommend using semver ranges in a safe manner
if [ "$semver_range" == "null" ]; then
protip "Specify a node version in package.json"
semver_range=""
elif [ "$semver_range" == "*" ]; then
protip "Avoid using semver ranges like '*' in engines.node"
elif [ ${semver_range:0:1} == ">" ]; then
protip "Avoid using semver ranges starting with '>' in engines.node"
fi
# Output info about requested range and resolved node version
if [ "$semver_range" == "" ]; then
status "Defaulting to latest stable node: $node_version"
else
status "Requested node range: $semver_range"
status "Resolved node version: $node_version"
fi
# Download node from Heroku's S3 mirror of nodejs.org/dist
status "Downloading and installing node"
node_url="http://s3pository.heroku.com/node/v$node_version/node-v$node_version-linux-x64.tar.gz"
curl $node_url -s -o - | tar xzf - -C $build_dir
# Move node into ./vendor and make it executable
mkdir -p $build_dir/vendor
mv $build_dir/node-v$node_version-linux-x64 $build_dir/vendor/node
chmod +x $build_dir/vendor/node/bin/*
PATH=$PATH:$build_dir/vendor/node/bin
# Run subsequent node/npm commands from the build path
cd $build_dir
# If node_modules directory is checked into source control then
# rebuild any native deps. Otherwise, restore from the build cache.
if test -d $build_dir/node_modules; then
status "Using existing node_modules directory"
status "Rebuilding any native dependencies"
npm rebuild 2>&1 | indent
elif test -d $cache_dir/node_modules; then
status "Restoring node_modules from cache"
cp -r $cache_dir/node_modules $build_dir/
fi
NODE_VERSION=$(package_resolve_version "node")
echo "Using Node.js version: ${NODE_VERSION}" | indent
NPM_VERSION=$(package_resolve_version "npm")
echo "Using npm version: ${NPM_VERSION}" | indent
# cache directories
CACHE_STORE_DIR="$CACHE_DIR/node_modules/$NODE_VERSION/$NPM_VERSION"
CACHE_TARGET_DIR="$BUILD_DIR/node_modules"
# s3 packages
NODE_PACKAGE="http://${S3_BUCKET}.s3.amazonaws.com/nodejs-${NODE_VERSION}.tgz"
NPM_PACKAGE="http://${S3_BUCKET}.s3.amazonaws.com/npm-${NPM_VERSION}.tgz"
SCONS_PACKAGE="http://${S3_BUCKET}.s3.amazonaws.com/scons-${SCONS_VERSION}.tgz"
# vendor directories
VENDORED_NODE="$(mktmpdir node)"
VENDORED_NPM="$(mktmpdir npm)"
VENDORED_SCONS="$(mktmpdir scons)"
# download and unpack packages
echo "-----> Fetching Node.js binaries"
package_download "nodejs" "${NODE_VERSION}" "${VENDORED_NODE}"
package_download "npm" "${NPM_VERSION}" "${VENDORED_NPM}"
package_download "scons" "${SCONS_VERSION}" "${VENDORED_SCONS}"
# vendor node into the slug
PATH="$BUILD_DIR/bin:$PATH"
echo "-----> Vendoring node into slug"
mkdir -p "$BUILD_DIR/bin"
cp "$VENDORED_NODE/bin/node" "$BUILD_DIR/bin/node"
# setting up paths for building
PATH="$VENDORED_SCONS:$VENDORED_NODE/bin:$PATH"
INCLUDE_PATH="$VENDORED_NODE/include"
export CPATH="$INCLUDE_PATH:$CPATH"
export CPPPATH="$INCLUDE_PATH:$CPPPATH"
# install dependencies with npm
echo "-----> Installing dependencies with npm"
run_npm "install --production"
run_npm "rebuild"
echo "Dependencies installed" | indent
echo "-----> Building runtime environment"
mkdir -p $BUILD_DIR/.profile.d
echo "export PATH=\"\$HOME/bin:\$HOME/node_modules/.bin:\$PATH\"" > $BUILD_DIR/.profile.d/nodejs.sh
# Make npm output to STDOUT instead of its default STDERR
status "Installing dependencies"
npm install --production 2>&1 | indent
status "Pruning unused dependencies"
npm prune 2>&1 | indent
status "Caching node_modules directory for future builds"
rm -rf $cache_dir
mkdir -p $cache_dir
test -d $build_dir/node_modules && cp -r $build_dir/node_modules $cache_dir/
status "Cleaning up node-gyp and npm artifacts"
rm -rf "$build_dir/.node-gyp"
rm -rf "$build_dir/.npm"
# Update the PATH
status "Building runtime environment"
mkdir -p $build_dir/.profile.d
echo "export PATH=\"\$HOME/vendor/node/bin:\$HOME/bin:\$HOME/node_modules/.bin:\$PATH\"" > $build_dir/.profile.d/nodejs.sh
# Post package.json to nomnom service
# Use a subshell so failures won't break the build.
(
curl \
--data @$build_dir/package.json \
--fail \
--silent \
--request POST \
--header "content-type: application/json" \
https://nomnom.heroku.com/?request_id=$REQUEST_ID
) &
#!/usr/bin/env bash
# bin/release <build-dir>
cat <<EOF
---
config_vars:
PATH: bin:node_modules/.bin:/usr/local/bin:/usr/bin:/bin
EOF
#!/usr/bin/env bash
#
# Create a Heroku app with the following buildpack:
# https://github.com/ddollar/buildpack-test
#
# Push this Node.js buildpack to that Heroku app to
# run the tests
#
# See CONTRIBUTING.md for info on running these tests.
testDetectWithPackageJson() {
detect "package-json-version"
detect "stable-node"
assertCaptured "Node.js"
assertCapturedSuccess
}
......@@ -19,48 +12,113 @@ testDetectWithoutPackageJson() {
assertCapturedError 1 ""
}
testPackageJsonWithVersion() {
compile "package-json-version"
assertNotCaptured "WARNING: No version of Node.js specified"
assertCaptured "Using Node.js version: 0.6.11"
assertCaptured "Using npm version: 1.1.9"
testNoVersion() {
compile "no-version"
assertCaptured "PRO TIP: Specify a node version in package.json"
assertCaptured "Defaulting to latest stable node"
assertCapturedSuccess
}
testPackageJsonWithoutVersion() {
compile "package-json-noversion"
assertCaptured "WARNING: No version of Node.js specified"
assertCaptured "Using Node.js version: 0.10"
assertCaptured "Using npm version: 1.3"
testDangerousRangeStar() {
compile "dangerous-range-star"
assertCaptured "PRO TIP: Avoid using semver ranges like '*'"
assertCaptured "Requested node range: *"
assertCaptured "Resolved node version: 0.10"
assertCapturedSuccess
}
testPackageJsonWithInvalidVersion() {
compile "package-json-invalidversion"
assertCapturedError 1 "Requested engine npm version 1.1.5 does not"
testDangerousRangeGreaterThan() {
compile "dangerous-range-greater-than"
assertCaptured "PRO TIP: Avoid using semver ranges starting with '>'"
assertCaptured "Requested node range: >"
assertCaptured "Resolved node version: 0.10."
assertCapturedSuccess
}
testNothingCached() {
cache=$(mktmpdir)
compile "package-json-version" $cache
testRangeWithSpace() {
compile "range-with-space"
assertCaptured "Requested node range: >= 0.8.x"
assertCaptured "Resolved node version: 0.10."
assertCapturedSuccess
}
testStableVersion() {
compile "stable-node"
assertNotCaptured "PRO TIP: Avoid using semver"
assertNotCaptured "PRO TIP: Specify"
assertCaptured "Resolved node version"
assertCapturedSuccess
}
testUnstableVersion() {
compile "unstable-version"
assertCaptured "Requested node range: >0.11.0"
assertCaptured "Resolved node version: 0.11."
assertCapturedSuccess
assertEquals "0" "$(ls -1 $cache | wc -l)"
}
testProfileCreated() {
compile "package-json-version"
compile "stable-node"
assertCaptured "Building runtime environment"
assertFile "export PATH=\"\$HOME/bin:\$HOME/node_modules/.bin:\$PATH\"" ".profile.d/nodejs.sh"
assertFile "export PATH=\"\$HOME/vendor/node/bin:\$HOME/bin:\$HOME/node_modules/.bin:\$PATH\"" ".profile.d/nodejs.sh"
assertCapturedSuccess
}
testInvalidDependency() {
compile "invalid-dependency"
assertCaptured "not in the npm registry"
assertCapturedError 1 ""
}
testNodeModulesCached() {
cache=$(mktmpdir)
compile "caching" $cache
assertCaptured "Caching node"
assertEquals "1"