Posting notebooks to WordPress

WordPress is one of the more popular content management systems out there. It powers more than 60 million websites. This website is one of them, as is the official Wolfram Research blog. In this post, I will present a Mathematica package that can be used to interact with WordPress websites programmatically. I started writing this several years ago as I felt that it would be very nice to be able to turn notebooks into WordPress posts seamlessly.

The solution consists of four packages. There is a package for creating XML-RPC requests because WordPress' API is an XML-RPC API. There is a package that provides Mathematica functions corresponding to the types of requests that can be made to WordPress, such as uploading a file or creating a new post. There is a package for converting Mathematica notebooks into HTML suitable for WordPress. And finally, there is a package that provides a graphical user interface for posting content to WordPress from Mathematica, in the form of a palette.

The solution also consists of some JavaScript code since one of the most interesting features is that it can turn code into copyable images. What that means is that it turns a notebook cell into an image. When the user clicks on the image, the corresponding code is copied into the clipboard. This is a feature also seen on several of Wolfram Research's websites.

The code is available as open source and the rest of this article will be about how to install this so that you can use it yourself. By reading this article, you will also better understand what the packages do.

Installing the Wolfram Language XML-RPC client

The packages are distributed in the form of a paclet. To install them, go to the releases page and download the latest version of the paclet, then open a Mathematica notebook and evaluate the following code. Make sure you have updated the file path:

Configuring the palette

After you have installed the paclet, the palette will automatically appear under the Palettes menu. How to configure the palette is pretty self-explanatory. Here is an image of how it looks:

You press the buttons to open the configuration dialogs and you have to press save to make the changes. The XML-RPC API endpoint is usually yourwebsite.com/xmlrpc.php. This is the address to which API requests will be sent. The username and password should correspond to your WordPress user, and if you want to upload things that user needs to have those privileges (nothing special, you already have them if you can write posts in the browser.) You can press the verify button to check that your connection with WordPress works. If the endpoint is wrong it will tell you that, if your endpoint is right but your username or password is wrong it will also tell you that. Otherwise, it will tell you that it can connect to WordPress.

"Code appearance" in the settings dialog means whether you want to post code as images or as text when you upload a notebook. "Mathematica Toolbox text code highlighting" means that if you upload code as text, should the markup be plain HTML that works always, or should it be using the WordPress shortcodes provided by the Mathematica Toolbox WordPress plugin. The advantage with using Mathematica Toolbox is that you then get the same code highlighting as is used on Wolfram Community and Mathematica Stack Exchange. You can also decide whether it should group consecutive input-output groups into one image, or if each such group should be uploaded as a separate image. The advantage of grouping is primarily that the reader only has to click once to copy the input from all the input cells. Finally, we can set the page width that is used by the notebook when rasterizing expressions. This affects how Mathematica breaks code that's too wide to display on one line. I set this to the width of the text on the website, which here on Wolfram Language Reviews is 720 pixels.

Uploading notebooks

A notebook can be uploaded by opening the palette, selecting the notebook, and then pressing the "Notebook" button. A preview at the top of the dialog that's opened up shows you the top of the selected notebook, so you can be sure that you selected the right one:

The target field can be used to specify an ID of a WordPress post. If you set it and then press the "Set target" button it will fetch all the settings from that post. If you then press "Post notebook" it will overwrite the content of the post with that ID rather than uploading the notebook as a new post. If you have uploaded the selected notebook before then its WordPress post ID will already be stored in the notebook using tagging rules, and the dialog will be pre-filled with its information.

Uploading cells

The "Image" button can be used to upload individual cells or groups of cells. Just select the cell and press the button and you will see a preview like this:

The HTML code for inserting the image into WordPress will be copied to the clipboard. If the "make input copyable" checkbox is checked then it will also copy the HTML needed to support this feature. If WordPress is properly configured, it will then look like this in the browser:

It is possible to also select text cells, individual images, or pretty much anything else in the notebook to upload in this way.

Configuring WordPress

In order to make copyable images work, we need to do some things on the WordPress side as well. Specifically, we need to add some JavaScript and some CSS and some PHP to make it load the JavaScript and CSS. There are different ways of adding this code to your WordPress theme. The most correct way is to create a child theme and then put the additional code in that. This way, when the parent theme updates it won't affect the changes that you've made. It also works to modify the theme directly, however, if you don't intend to use the update mechanism or if you do it so rarely that you are willing to restore the changes if they are removed.

The first thing you need to do is to download the code from the Github repository. In the resources folder, you'll find wlxmlrpcclient.css, wlxmlrpcclient.js, and functions.php. You need to put the CSS file and the JavaScript file in the root folder of your theme. Furthermore, you need to open the functions.php file and copy its content into the functions.php that should already be present in your theme. The PHP file contains the code necessary to load the JavaScript and CSS, but it also contains the following code:

remove_filter('the_content', 'wptexturize');
remove_filter('the_excerpt', 'wptexturize');
remove_filter( 'the_content', 'wpautop', 99 );
remove_filter( 'the_excerpt', 'wpautop', 99 );

It has some undesireable side-effects so it is important to understand what it does. The two first lines make sure that WordPress doesn't run the wptexturize filter on the main text or the post excerpt. The last two lines make sure that WordPress doesn't run the wpautop filter on the main text or the excerpt. The wptexturize filter takes a text as input and outputs the same text with some characters replaced, among other things it replaces quotes with smart quotes. This is not good for our copyable code that we've inserted into the post, because if quotes are replaced with smart quotes then the Wolfram Language code breaks. That's why we're preventing that. The wpautop filter takes a text as its input and returns a new text with double line breaks replaced with a block of text surrounded by paragraph HTML tags. This also doesn't work well with code because it can insert HTML paragraph tags into the code.

The problem with disallowing these filters is that things will not work as you expect anymore. Especially because of the removal of wpautop. What I do when I write new posts without wpautop is that I go into text mode and write the post in HTML instead of using the WYSIWYG editor. I add the paragraph tags (<p></p>) manually. I'm careful not to go back to WYSIWYG mode because that will remove my tags. But note that if you are uploading a notebook you don't need to do this: the package adds the paragraph tags for you. If you only use the package to post content, then, you don't need to think about it.

Realistically speaking, uploaded notebooks are not your only content, however. It can be especially problematic that the removal of these filters will also affect the formatting of previously written posts. Given this, I would suggest using the WordPress plugin Raw HTML. It can turn off these filters on a per-post basis. Use it, and only turn the filters off on the posts that you uploaded from notebooks. Or if you are writing a post by uploading images of cells with copyable code one by one using the image button.

Update (13/2/2019): Disabling the wpautop filter is not possible with the new Gutenberg editor in WordPress 5. I currently use the Disable Gutenberg plugin to make it work.

Known problems and limitations

It is mostly alright to think about the notebook upload function as if it can really upload a notebook. However, it should be remembered that it is only made to be able to convert certain types of cells into HTML. It can deal with font styles, font color, heading cells, text cells, images, input, output, and more, but with such a flexible format such as a notebook there are many types of cells and features that it doesn't support. When it encounters these cells that it doesn't know how to handle it is supposed to just skip them. Please keep your notebooks basic if you intend to upload them with the package (e.g. by using the palette.)

When I started using the palette to upload notebooks for the tutorials on this site, I noticed that it tends to break down when the notebooks are large. It starts uploading images one by one but after a seemingly random number of images, it just stops. I have spent quite some time trying to debug this. I have an idea for refactoring that might solve it, but it is quite difficult to do this refactoring. I don't have a timeline for when this will be fixed. Since I use the package myself, I have incentive to fix this problem. What I currently do if I can't upload the notebook is that I upload code cell by code cell using the image button in the palette, similar to how I would write Mathematica Stack Exchange posts. It means I have to write the paragraph tags myself as well in the text mode of WordPress, as I explained earlier, but I find that it works quite well.

The future

I'm always interested in hearing how people are using this package. Feedback could potentially lead to new features.

About the author

Calle Ekdahl

Mathematica enthusiast from Gothenburg, Sweden.

Add comment

By Calle Ekdahl