From Ghost to 11ty

Over 5 years ago, I migrated this blog From Jekyll to Ghost. Back then, my reasons for moving to Ghost were because -

• I wanted a CMS
• Managing all the posts’ raw markdown files in one folder was getting clunky
• The Jekyll build process was starting to significantly lag
• I no longer wanted to have to manually upload new posts via FTP
• I wanted to be able to schedule posts (although Ghost doesn’t support this yet), or at least push one button to publish them

Today, I want the exact opposite 😅

• I no longer need a CMS. Since I now post a lot less frequently than before, having a full-blown CMS isn’t necessary.
• While raw markdown files can be a bit clunky, I prefer them to Ghost’s CMS. One of the reasons I chose Ghost in the first place was because, at the time, the editor allowed you to write in pure markdown. Over the years, they’ve made this more difficult and basically force you to use their WYSIWYG editor.
• The build process of 11ty is fast, even with 160+ articles.
• Luckily, the world has progressed from FTP 🥳 . Now, I can publish on Netlify just by pushing to GitHub.
• While I haven’t yet set this up, scheduling blog posts is possible with 11ty and Netlify.

So, last week, I migrated this blog to 11ty. I managed to do the bulk of the migration in one day, but continued fixing some bugs over the next few days. Here’s how the migration process went.

Step 1 - Convert the Ghost export to markdown #

One of the great things about Ghost is it allows you to export all your site data into one big JSON file. So, the first thing to do in this migration was to convert that JSON export to individual markdown files for each post.

Luckily, there’s an NPM package for that - ghost-to-md. I did run into a slight issue with that package due to the library it was using to convert HTML content to Markdown. So, I created a fork of the package (ireade/ghost-to-md), using a different HTML-to-Markdown converter, which worked better for me.

Using this package, I was able to generate a folder filled with individual markdown files for each blog post.

Step 2 - Organise each post into sub-folders #

With 11ty, you have a lot of freedom with how and where you want to store all post files. For me, I wanted a structure where, within the directory for all posts, each post will have its own folder, and the markdown file will be within that folder. So, instead of a folder structure like this -

- content
  - blog
    - article.md

I'd have a structure like this -

- content
  - blog
    - article
      - article.md
      - [other files related to the article]

The reason I wanted it this way was so that within each article folder, I could also store all the media related to that article. This makes it easy to use the 11ty shortcode for responsive images (more on that in Step 4).

To achieve this, I wrote a script to read all the exported markdown files, and create new subdirectories for each of them.

const inputDir = './ghost-to-md-output';
const outputDir = './output';

// Read all files in inputDir
fs.readdir(inputDir, (err, files) => {
  if (err) return;

  // Loop through each file
  files.forEach((file) => {

    const fileName = path.parse(file).name;
    const inputFilePath = path.join(inputDir, file);
    const outputFolderPath = path.join(outputDir, fileName);

    // Create folder in outputDir
    fs.mkdir(outputFolderPath, err => {
      if (err && err.code !== 'EEXIST') return;

      const outputFilePath = path.join(outputFolderPath, file);

      // Copy markdown file to new folder
      fs.copyFile(inputFilePath, outputFilePath, err => {
        if (err) return;

        console.log(`Successfully copied ${file} to ${outputFolderPath}`);
      });
    }); // end fs. fs.mkdir
  }); // end files.forEach
});

The result was something like this -

Step 3 - Download media hosted on Ghost #

The next step was to download media content hosted on Ghost. If you look through the markdown files, you’ll notice that some of the URLs looks like this -

__GHOST_URL__/content/images/2019/01/image.png

This refers to images/media uploaded via the CMS and hosted on Ghost. The JSON export doesn’t include any media, so what I needed to do was manually download every image and place them in the correct folder.

Again, I used a script for this. I created a function that:

1. Takes the folder path to an article, plus the article file name;
2. Downloads all images starting with __GHOST_URL__; *
3. Places them in the articleFolderPath;
4. Replaces the URL in the markdown file with a local relative URL, (i.e. starting with ./)

// Takes the folder path to an article, plus the article file name
function downloadImages(articleFolderPath, fileName) {

  const markdownFilePath = articleFolderPath + "/" + fileName;

  fs.readFile(markdownFilePath, 'utf8', (err, data) => {
    if (err) return;

    //  🚨Warning - Very buggy! (see note below)
    // Find all images starting with __GHOST_URL__
    const regex = /\(__GHOST_URL__([\s\S]*?)\)/g;
    let match;

    while ((match = regex.exec(data)) !== null) {

      const ghostHostedUrl = 'https://bitsofco.de' + match[1];
      const filename = path.basename(ghostHostedUrl);
      const filePath = path.join(articleFolderPath, filename);

      console.log(`Downloading image from ${ghostHostedUrl} to ${filePath}`);

      const file = fs.createWriteStream(filePath);
      https.get(ghostHostedUrl, response => {
        response.pipe(file);
        console.log(`Successfully downloaded image to ${filePath}`);
      }).on('error', err => {
        console.log(`Error downloading image: ${err}`);
        fs.unlink(filePath, () => {});
      });

      // Replaces the URL in the markdown file with a local relative URL
      data = data.replace(match[0], `(./${filename})`);
    }

    fs.writeFile(markdownFilePath, data, err => {
      if (err) return;
      console.log(`Successfully updated file: ${markdownFilePath}`);
    });
  });

}

* Note - This regex was buggy. I later realised that there were more URLs that started with this same __GHOST_URL__ string. For example, links to other pages within the Ghost site started with that string, so the script replaced those instances as well. If you want to use this, I'd suggest you update the regex to only look for images.

Step 4 - Use the 11ty Image Plugin #

Since each blog post is written in markdown, I could link to images using the regular markdown syntax -

![Image description](image.png)

However, 11ty has an Image Plugin, which performs build-time image transformations for locally-stored images. By linking to one single image file, the plugin will automatically generate images in different sizes and formats. To take advantage of this, instead of using the markdown syntax above, we need to use this image shortcode -

{% image "image.png", "Image description" %}

So, I wrote a script to make this change. This script was also pretty buggy and resulted in the shortcode being inserted into places is shouldn’t have, so I wouldn’t recommend you use as-is. That said, it did work well enough for me, I just had to go through and fix some buggy places.

function convertMarkdownImagesTo11ty(articleFolderPath, fileName) {

  const markdownFilePath = articleFolderPath + "/" + fileName;

  fs.readFile(markdownFilePath, 'utf8', (err, data) => {
    if (err) return;

    //  🚨Warning - Very buggy!
	//  Find all images starting with "./"
    const regex = /!\[(.*?)\]\(\.\/(.*?)\)/g;
    let match;

    while ((match = regex.exec(data)) !== null) {
      const altText = match[1];
      const imagePath = match[2];
      const eleventyImage = `{% image "${imagePath}", "${altText}" %}`;
      data = data.replace(match[0], eleventyImage);
    }

    fs.writeFile(markdownFilePath, data, err => {
      if (err) return;
      console.log(`Successfully updated file: ${markdownFilePath}`);
    });
  });
}

Step 5 - Setup 11ty with eleventy-base-blog #

Now that the content was all set up, I could finally move on to 11ty! This part was pretty straightforward. To setup the new project, I used the 11ty/eleventy-base-blog starter repository. This already had setup all the relevant pages and templates. All I had to do was tweak the page structure and styling to the theme of my blog.

To add all my post content, I just pasted all the newly-created directories into the content/blog directory.

A few implementation details to mention -

Permalinks #

The URL structure for the articles on Ghost was pretty simple -

https://bitsofco.de/[article-slug]

Luckily, the ghost-to-md script does include the post slug in each file’s front matter. So, all I had to do was make it such that all posts used that slug for their permalink. To do that, I added the following to the content/blog/blog.11tydata.js file, which has global configuration for all posts.

module.exports = {
	/* other config */
	eleventyComputed: {
		permalink: data => `/${data.page.fileSlug}/`
	}
};

Tags #

One issue with the conversion from Ghost to Markdown is how tags are handled. In the export, multiple tags on a post are comma-separated, like this -

---
title: Example Article
slug: example-article
tags: one, two
---

However, 11ty needs them to be formatted like this -

---
title: Example Article
slug: example-article
tags:
  - one
  - two
---

This was an annoying bug to find out at the 11th hour, and I ended up having to fix it manually. If I were to do this again, I'd probably write a script to automate the process.

Update - An issue has been filed to support tags as a comma-separated list! (#2820)

Post excerpts #

As far as I know, 11ty doesn’t have a native way to show excerpts from posts, so I had to add a custom shortcode to do that. I got the code for that from this article - Excerpts with Eleventy, by Jonathan Yeong.

CanIUse Embed #

Since the embed I created to display caniuse data is an integral part of my posts, I wanted to create a custom plugin to make embedding them easier (similar to how the image plugin works). Luckily for me, someone already beat me to it 😅 . I integrated the plugin, KevinGimbel/eleventy-plugin-caniuse, and now I can add the embed by using the following shortcode -

{% caniuse "css-grid" %}

And get this result -

Set 6 - Run npm start and fix bugs #

After building the 11ty site, a lot of random bugs came up that I had to fix manually. I was able to debug them pretty easily due to the descriptive error messages, but here are some of the main ones in case you run into similar issues -

• Markdown errors, especially with tables - This wasn't really an error but rather the markdown tables were sometimes strange. If you're going through this migration process, I would recommend that you go through all the places you have tables to make sure they look right.
• Using nunjucks within the code examples - One weird bug I ran into was if I happened to be demonstrating a code example within my markdown file that was in Nunjucks. For example, the code above demonstrating how I use the caniuse shortcode. In these cases, I had to wrap the code block with a {% raw %}{% endraw %} to make sure it's not trying to execute the block itself.
• Double-quotes in post titles - Another thing to look out for is if quotes are being used in the front-matter. In some cases, you may need to escape the quotes by adding a backwards slash \" in front of them
• Broken links - I also did a check for broken links using this free broken link checker. This helped me track down and fix those random side-cases, a lot of which were closed by my own code above 😂

 

That was it! It’s only been a few days, but so far I’m loving 11ty. I'm also in the middle of doing a redesign, so more on that soon!