Using Python functions in Node.js

Boa’s main purpose is to allow you to use Python interfaces and functions in Node.js. Maybe you will be curious, confused, and puzzled when I see it here. My Node.js magic is so good. Why use Python? If you have tried some applications of JavaScript for machine learning before, you will understand the reason behind this.

The current situation is that the machine learning ecosystem is almost tied to Python. The language is iterating at high speed, and JavaScript can only count on its own. If we expect to achieve Python’s current scale from zero, the amount of work required is huge. When I wrote tensorflow-nodejs years ago, I already thought so.

Therefore, we must change our thinking. Since we can’t surpass Python, then will use Python. For developers of scripting languages, they don’t care how the low-level the layer is implemented, as long as the high-level language and interface are familiar to me, so Boa is a Node.js library born for this, which bridges CPython to provide JavaScript with the ability to access the complete Python ecosystem, and in addition, with the help of ES6 new features to provide users with a seamless development experience, so what is the experience?

Let’s look at a simple example:

  1. const boa = require('@pipcook/boa');  
  2. const os = boa.import('os');  
  3. console.log(os.getpid()); // prints the pid from python.  
  4. // using keyword arguments namely \`kwargs\`  
  5. os.makedirs('..', boa.kwargs({  
  6.   mode: 0x777,  
  7.   exist\_ok: false,  
  8. }));  
  9. // using bult-in functions  
  10. const { range, len } = boa.builtins();  
  11. const list = range(0, 10); // create a range array  
  12. console.log(len(list)); // 10  
  13. console.log(list\[2\]); // 2

Isn’t it simple enough? Just load the Python object through boa.import and the remaining object access, a function call, and array access are no different from our JavaScript.

  1. const boa = require('@pipcook/boa');  
  2. const tf = boa.import('tensorflow');  
  3. const { layers, Model } = tf.keras;
  5. class TestModel extends Model {  
  6.   constructor() {  
  7.     super();  
  8.     this.conv1 = layers.Conv2D(32, 3, boa.kwargs({ activation: 'relu' }));  
  9.     this.flatten = layers.Flatten();  
  10.     this.d1 = layers.Dense(128, boa.kwargs({ activation: 'relu' }));  
  11.     this.d2 = layers.Dense(10, boa.kwargs({ activation: 'softmax' }));  
  12.   }  
  13.   call(x) {  
  14.     return this.conv1(x)  
  15.       .flatten(x)  
  16.       .d1(x).d2(x);  
  17.   }   
  18. }

The above example shows how to use TensorFlow to create a model, in addition to demonstrating how to inherit from a Python class from JavaScript. Is this a very JavaScript style?

It is worth mentioning that, in the Boa internals, there is no encapsulation of TensorFlow and other frameworks, as long as you install the corresponding package through Python locally, it can be used as the above code, so in theory you can use any Python package to do what is done above.

See examples/boa for more examples that is written in Boa.

Overview

If you are a plugin developer, then you do not need any installation operation, Pipcook will install Boa for you. Or you want to use it independently? You can install it via npm:

  1. $ npm install @pipcook/boa

Next, we will introduce some of the main methods.

builtins()

Python will build some common functions in its builtins, the specific API list is at: https://docs.python.org/3.7/library/functions.html, then Boa also provides corresponding way to use them:

  1. const { len, list, range } = boa.builtins();

import(name)

In addition to the built-in methods, the most important function is to load Python packages, so import is to do this.

  1. const np = boa.import('numpy');

kwargs(map)

Next is the keyword arguments. In Python, there is a way to use a map to represent parameters, such as:

  1. foobar(100, x=10, y=20)

It helps the caller understand the meaning of each parameter better. For this reason, the kwargs method has been added to Boa to support this usage:

  1. foobar(100, boa.kwargs({ x: 10, y: 20 }));

with(ctx, fn)

“with” may be familiar to some people who are familiar with the history of JavaScript, but “with” in Python, its usage, and its purpose are not the same as JavaScript. The with-statement in Python is a bit similar to Block Scoping in JavaScript:

  1. with(localcontext()) {  
  2.   \# balabala  
  3. }

The above code saves the state of localcontext(), then starts executing the block code in the with-statement, and finally, release the state of localcontext().

The internal implementation mechanism is that each variable passed into the with-statement needs to implement two magic methods: __enter__ and __exit__, and then called before and after the block code execution.

eval(str)

The last to tell is to evaluate Python expressions (single line). Why should we provide such a method? This still has to talk about the advantages of Python. In some very complex data processing scenarios, Python expressions can still be expressed very simply and understandably, which greatly reduces the complexity of the code. Let’s take a look at an example:

  1. const line = (boa.eval\`'\\t'.join(\[str(x) for x in ${vec}\])\`);

If the above code is to be replaced with JavaScript:

  1. vec.map(x => x.toString()).join('\\t');

How much does it seem to be almost right? Then take a look at the following example:

  1. boa.eval\`{u:i for i, u in enumerate(${vocab})}\`;  
  2. boa.eval\`\[${char2idx}\[c\] for c in ${text}\]\`;
  3. boa.eval\`${chunk}\[:-1\]\`;
  4. boa.eval\`${chunk}\[0:-1:2\]\`;

How about it, does it feel like the above example can’t be done with a simple line of JavaScript?

However, it is worth mentioning that JavaScript is gradually making up in this regard. there are some related standards that TC39 is doing, including the above Slice Notation.

Speaking of returning to eval, it is like a supplement to JavaScript. Before some standards have been implemented and stabilized, it allows us to use Python expressions to express more simply, and all we need is some low-cost learning. That’s it.

Next, let’s talk about how eval is used. It accepts a “string”, but we generally pass Template String when using it. Let’s look at two examples first:

  1. boa.eval('print("foobar")');
  2. boa.eval(\`print("${txt}")\`);

After reading the above 2 lines of code, they are relatively rare usages. The most commonly used and the most effective eval is the use of Tagged Template String. This usage is just like what we saw at the beginning. The content of the template string is directly followed by eval. The advantage of this is that the eval function will receive To all the template parameters so that we can open up JavaScript objects and Python expressions to achieve a smoother user experience, as follows:

  1. const chunk = range(0, 10);  
  2. boa.eval\`${chunk}\[0:-1:2\]\`;

The above is to transfer the chunk to the expression, and then get the corresponding value through the Slice Notation syntax of Python, and finally return to the world of JavaScript.

Summary

Well, the simple API introduction will come here first. If you want to learn more about API and Boa, you can go to our guide: Introduction to Boa.

Finally, let’s talk about the original intention of Boa, which is to allow JavaScript developers to use Python’s rich machine learning ecosystem more seamlessly. It can be said that starting today, you can start looking at Python documentation and use JavaScript to “learn and use” machine learning and deep learning!