openSUSE:Packaging nodejs
Build Service Tutorial · Tips & Tricks · Cross Distribution Howto · Packaging checks
Desktop menu categories · RPM Macros · Scriptlets · Init scripts · How to write good changes
Please refer to this article's discussion page for more information.
NPM
Very much like CPAN for Perl/PYPI for python, nodejs has a smiliar archive network, hosting nodejs modules, which is npm. Also npm is the pip-in-python like command in nodejs to install modules. Unlike python, most of the nodejs modules don't have any official website. This fact makes npmjs.org the best place to find latest versions and development porals for nodejs modules.
Devel repository
All nodejs packages in openSUSE are developed at devel:languages:nodejs repository on Open Build Service.
A simple example using npm installation
Most of the nodejs modules can be installed through npm, with some still using the legacy `configure && make && sudo make install` method, which is rather rare.
The example below are taken from the nodejs-minimist package in devel:languages:nodejs repository.
NOTE: This method can only be used for nodejs modules that have no dependency on any other nodejs modules. Read below.
#
# spec file for package nodejs-minimist
#
# Copyright (c) 2014 SUSE LINUX Products GmbH, Nuernberg, Germany.
#
# All modifications and additions to the file contributed by third parties
# remain the property of their copyright owners, unless otherwise agreed
# upon. The license for this file, and modifications and additions to the
# file, is the same license as for the pristine package itself (unless the
# license for the pristine package is not an Open Source License, in which
# case the license is the MIT License). An "Open Source License" is a
# license that conforms to the Open Source Definition (Version 1.9)
# published by the Open Source Initiative.
# Please submit bugfixes or comments via http://bugs.opensuse.org/
#
Name: nodejs-minimist
Version: 0.2.0
Release: 0
License: MIT
Summary: Parse argument options
Url: https://github.com/substack/minimist
Group: Development/Libraries/Other
Source: minimist-%{version}.tar.gz
BuildRequires: nodejs
BuildRequires: nodejs-packaging
BuildRoot: %{_tmppath}/%{name}-%{version}-build
BuildArch: noarch
%{?nodejs_requires}
%description
This module is the guts of optimist's argument parser
without all the fanciful decoration.
%prep
%setup -q -n minimist-%{version}
%build
%install
%nodejs_install
rm %{buildroot}%{nodejs_modulesdir}/minimist/LICENSE
rm %{buildroot}%{nodejs_modulesdir}/minimist/readme.markdown
%files
%defattr(-,root,root)
%doc LICENSE readme.markdown
%{nodejs_modulesdir}/minimist
%changelog
Naming scheme
nodejs packages in openSUSE are named like Python's and Perl's, starting with `nodejs-`.
Note: the downloaded source tarball might be in node-mkdirp-version.tar.gz or minimist-version.tar.gz format, so it requires to use -n to explicitly specify the unpack directory in the %setup process:
%setup -q -n node-mkdirp-%{version}
Most of the nodejs modules are architecture-independent
Because they are JavaScript instead of C, so like python, "pure" nodejs package are all arch-independent, which requires this line in the specfile:
BuildArch: noarch
nodejs RPM macros
All nodejs related RPM macros are inside /etc/rpm/macros.nodejs and /etc/rpm/macros.coffeescript. Most common:
%{?nodejs_requires}
Use as Requires, equals to Requires: nodejs.
%{nodejs_modulesdir}
Equals to /usr/lib/node_modules.
%nodejs_install
Equals to:
mkdir -p %{buildroot}%{nodejs_modulesdir} \
npm_config_prefix=%{buildroot}%{_prefix} npm install -g %{S:0}
We have to talk about npm again here. NPM can install from online URL or from local file. With a specified prefix, it will install stuff into the ./usr/lib/node_modules directory, which is %{nodejs_modulesdir}, of our Buildroot. The '-g' option means "Globally install" instead of "Locally by user", which will install into /usr/local. %{S:0} is a shortname for %{SOURCE0}, which is the content of the Source tag normally.
%cake_build
`cake build` in coffeescript.
%cake_install
`cake --prefix %{buildroot}%{nodejs_modulesdir} install` in coffeescript, will install into /usr/lib/node_modules.
NPM "dependencies" issues in package.json
Every nodejs package that can be installed through npm must have a package.json file inside its tarball. Actually, it is a npm instruction guides npm to install the package. Here is the problem:
If the package.json has a tag "Dependencies" with some other nodejs packages as dependency, the build process on OBS will certainly fail.
This reason is a combination of the two below:
- the build virtual machine on OBS does not have any network (so as to ensure the Island Test succeeds), unlike AUR.
- npm needs to check registry.npmjs.org for dependencies anytime, no matter whether you have had the dependencies installed or not. If you once have installed nodejs modules locally, you will find the "└── minimist@0.0.8"-like symbol, this is just the symbol that proves npm does query stuff online no matter whether you have it or not (because our minimist version is 0.2.0).
That is why you can install on your machine, or build a rpm on your machine, but cannot do the same thing on OBS.
Here are the dirty workarounds:
- Install the package on your own machine, and use `install -d <nodejs_modulesdir>; install -m 0644 <file>` method (Manual Copying) on OBS. but considering npm is a common method, such a workaround seems low...
- Like this:
- Unpack the tarball
- copy package.json to the top directory
- patch the package.json to remove the "Dependencies" tag
- npm install the patched directory
- install the original package.json to the BuildRoot (So end users can still have depdencies)
- Use the RPM requires to track the actual dependencies
An example (We may covert the procedures above to some nodejs RPM macros:
#
# spec file for package nodejs-mkdirp
#
# Copyright (c) 2014 SUSE LINUX Products GmbH, Nuernberg, Germany.
#
# All modifications and additions to the file contributed by third parties
# remain the property of their copyright owners, unless otherwise agreed
# upon. The license for this file, and modifications and additions to the
# file, is the same license as for the pristine package itself (unless the
# license for the pristine package is not an Open Source License, in which
# case the license is the MIT License). An "Open Source License" is a
# license that conforms to the Open Source Definition (Version 1.9)
# published by the Open Source Initiative.
# Please submit bugfixes or comments via http://bugs.opensuse.org/
#
Name: nodejs-mkdirp
Version: 0.5.0
Release: 0
License: MIT
Summary: Create Directory
URL: https://github.com/substack/node-mkdirp
Group: Development/Libraries/Other
Source: node-mkdirp-%{version}.tar.gz
Patch: node-mkdirp-remove-dependency-tag.patch
BuildRequires: nodejs
BuildRequires: nodejs-packaging
BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}
BuildArch: noarch
Requires: nodejs-minimist
%{?nodejs_requires}
%description
Create nested directorie, works like mkdir -p
%prep
%setup -q -n node-mkdirp-%{version}
cp -r package.json ..
%patch -p1
pushd ..
tar -czf %{SOURCE0} node-mkdirp-%{version}
popd
%build
%install
%nodejs_install
rm %{buildroot}%{nodejs_modulesdir}/mkdirp/LICENSE
rm %{buildroot}%{nodejs_modulesdir}/mkdirp/readme.markdown
# We do not need a binary that may look similiar with unix default ones.
rm -Rf %{_bindir}/mkdirp
# manual install original package.json
install -m 0644 ../package.json %{buildroot}%{nodejs_modulesdir}/mkdirp/
%files
%defattr(-,root,root)
%doc LICENSE readme.markdown
%{nodejs_modulesdir}/mkdirp
%changelog
Failed attempts
BuildRequires: xxx
It is a python thinking, "oh, setup.py said it needs something, so we give it by RPM BuildRequires and problem solved", but npm does not buy it.
npm link ./xxx
If you search "npm local dependency", you will find some "solutions" on StackOverflow said you can keep the "Dependencies" tag in package.json, while using:
cp -r %{nodejs_modulesdir}/minimist .
npm link ./minimist
to use your copy of minimist as local dependency. It will be installed into %{nodejs_modulesdir} together with your main package. Then, you can remove the dependency again to get the main package only in BuildRoot. Perfect theory!, but it will not work, because:
Before the link process, npm will at first unlink the one in your %{nodejs_modulesdir} (which means, npm can actually find the dependency you specified through RPM Requires), but the "abuild" user OBS used to control the build process does not have the permissions to do that.
npmrc or --registry
We tried to change npm registry to a local directory by using:
npm install --registry %{nodejs_modulesdir}
or:
npm config set regsitry %{nodejs_modulesdir}
all failed because:
[ 54s] npm WARN invalid config registry="/usr/lib/node_modules" [ 54s] npm WARN invalid config Must be a full url with 'http://'
The same thing also applys to --proxy http://localhost because anyway it will query some URL and return something.
"Manual copy" method vs. NPM method
Packages must be manually copied (Bootstraps for NPM)
These packages in d:l:nodejs must be updated with "manual copy" method, because they're build time depedencies for npm. As explained above, "manual copy" is really not a good way to package nodejs, although it's an option. But the NPM method will certainly need npm itself. So if a package, which is needed by npm to build npm, relies on npm to build itself, it will create a cycle that never ends. So we created these prerequisites plus their runtime dependencies with "manual copy" method so that we will not run into an endless loop.
These packages can not be broken by introducing npm BuildRequires. See this table below (ignored the "nodejs-" prefix"
| abbrev | ansi | ansicolors | ansistyles | archy | ansi-regex |
| block-stream | char-spinner | child-process-close | chmodr | chownr | clone |
| columnify | defaults | editor | inherits | strip-ansi | wcwidth |
| fstream | fstream-ignore | fstream-npm | github-url-from-git | github-url-from-username-repo | glob |
| graceful-fs | ini | inflight | lru-cache | minimist | mkdirp |
| once | rimraf | sigmund | wrappy | minimatch | asn1 |
| assert-plus | async | aws-sign2 | bl | boom | caseless |
| cmd-shim | combined-stream | delayed-stream | core-util-is | cryptiles | ctype |
| forever-agent | form-data | hawk | hoek | http-signature | init-package-json |
| isarray | json-stringify-safe | lockfile | mime | mime-db | mime-types |
| mute-stream | node-gyp | node-uuid | nopt | normalize-package-data | npmlog |
| oauth-sign | opener | osenv | promzard | punycode | qs |
| read | read-package-json | readable-stream | request | semver | sntp |
| string_decoder | stringstream | tar | tough-cookie | tunnel-agent | which |
| async-some | config-chain | debuglog | dezalgo | fs-vacuum | npm-cache-filename |
| npm-install-checks | npm-package-arg | npm-registry-client | npm-user-validate | util-extend | npm |
| npmconf | path-is-inside | proto-list | read-installed | readdir-scoped-modules | uid-number |
| retry | sha | slide | sorted-object | text-table | - |
(to be continued).
FAQ
npm ERR! network getaddrinfo enotfound
It is because your package has some dependencies on other nodejs packages in its package.json. Nodejs does not have a configure mechanism to check dependency issues before build (RPM dependencies do not work at all for NPM, BTW. That is why you should check package.json all the time before starting to package), so it can only stop when missing dependencies in `npm install` process, w/ a different, weird and totally wrong reason.
Once dependency problems are found during the npm installation process, npm will try to download the missing dependencies online, just like Python's pip. While unfortunate, even if the depdendencies are fulfilled by RPM Requires, it still needs to query registry.npmjs.org to "display" dependencies. But the build VM on openSUSE Build Service does not grant network connections, so fetching URL:
[ 110s] 160 http GET https://registry.npmjs.org/minimist [ 110s] 161 info retry will retry, error on last attempt: Error: getaddrinfo ENOTFOUND
will certainly fail, thus this getaddrinfo error occurs.
Dependencies for any nodejs module can be found under the 'Dependencies' tag (not the 'devDependencies' tag) in the package.json file inside its tarball, or from its page on npmjs.org. Also check #NPM "dependencies" issues in package.json for solutions.
How to debug npm issues
You need to replace %nodejs_install macro with:
npm_config_prefix=%{buildroot}%{_prefix} npm install --loglevel=silly -g %{S:0} || true
echo "=========================== LOG STARTED ============================="
if [ -f "npm-debug.log" ] ; then
cat npm-debug.log
elif
echo "all fine!"
done
echo "=========================== LOG ENDED ==============================="
And you can see anything npm did verbosely between those lines.