I've demonstrated how to create a todo list, in React, four times! Four-freaking-times! What if we don't want to use React? What if we want to use Vue? Or plain old HTML? What if we want to use React Native to build a mobile app?
I've got you covered fam. I am going to guide you through creating a client agnostic todo list API server with Node, Express, and Mongoose! That way you can use whatever front-end technology your heart desires.
See the completed code here. This tutorial assumes you have Node installed on your machine.
# create folder for the project and cd into itmkdir todo-list-api && cd $_
# create a read me for you repoecho "# Todo List API" >> README.mdInside the project, initiate Git and create a .gitignore file.
git initecho "node_modules" >> .gitignoreecho ".env" >> .gitignoreecho ".DS_Store" >> .gitignore # if using a MacNext, create a package.json file and use NPM to install our dependencies.
npm init -ynpm i express morgan nodemon dotenv mongooseexpress- web framework for Node that allows us to write handlers for HTTP requestsmorgan- a request logger for the consolenodemon- restarts the Node server when file changes are detecteddotenv- loads environment variables from a.envfilemongoose- an Object Data Modelling wrapper that provides a query API for interacting with a MongoDB database
Next add this script to the package.json to use Nodemon to fire up our server.
{ // ... "scripts": { "start": "nodemon server.js" } // ...}The server
Next let's create the simplest form of an Express server in server.js. We will require the top-level express() object and run listen() on it while we pass it a port number. We will also use the Morgan middleware to log our HTTP requests to the console.
const express = require('express');const morgan = require('morgan');
const port = process.env.PORT || 1337;const app = express();
app.use(morgan('dev'));
app.listen(port, () => console.log(`Server running on port ${port} 🚀`));Check to see if the server and Nodemon and working correctly.
npm run start# ...[nodemon] starting `node server.js`Server listening on port 1337 🚀MongoDB Atlas
Before we go any further we need to create a cluster on MongoDB Atlas. Here is the official tutorial on how to do so. A free, shared cluster will do for this tutorial. Once you've created the cluster, you'll need to get the database connection string in order to proceed.
Create an .env file at the root of your project. Add environment variables for the connection string and the port number. Replace <username>, <password> and <dbname> with the correct values in your own connection string.
DB_URL=mongodb+srv://<username>:<password>@cluster0.l9y8z9u.mongodb.net/<dbname>?retryWrites=true&w=majorityPORT=1337Connecting to MongoDB is fairly simple using mongoose.connect(). We just need to pass the function our connection string, which contains the path to our cluster and our credentials, and an options object, as arguments. Create a db.js file in ~/todo-list-api/config/.
const mongoose = require('mongoose');require('dotenv').config();
module.exports = async function () { try { await mongoose.connect(process.env.DB_URL, { useNewUrlParser: true, useUnifiedTopology: true, }); console.log('Successfully connected to database 🙌'); } catch (err) { console.log('Database connection unsuccessful: ', err); }};Now let's require our database connection function and invoke it.
const express = require('express');const morgan = require('morgan');const app = express();const db = require('./config/db');
app.use(morgan('dev'));
db();
app.listen(process.env.PORT, () => { console.log(`Server running on port ${process.env.PORT} 🚀`);});Nodemon should detect file changes, and reload the server for us. We should see in the console both our server running and that we've successfully connected to our MongoDB cluster in the cloud.
Server listening on port 1337 🚀Successfully connected to database 🙌Project structure
We will modularize our application according to the Route-Controller-Model-Services paradigm. There is no "right way" to structure your Node app, but this is considered pretty standard.
routes- the URL paths, each with it's own corresponding HTTP verb, and route handler - our API endpointscontroller- handles our HTTP requests, our parameter and query logic, as well as sending responses with the correct HTTP codes - where our route handler code livesmodel- the data-layer and model definitionservices- contains the database queries and return objects
We don't have a complicated API, so we will combine our controller and service layers.
.├── api│ └── todo│ └── model.js│ └── controller.js│ └── routes.js├── lib│ └── middleware.js├── config│ └── db.js├── node_modules├── package.json├── README.md├── .gitignore├── .env└── server.jsTodo model
In Mongoose, a schema defines the stucture of a document, it's default values, and validators. A model is a wrapper around on the schema which provides a set of methods for interacting with the database, such as querying and CRUD operations.
In model.js create the schema for our todo model. The schema is passed as the second argument to the mongoose.model() method, while the first argument is what we will name our model. Later on in the tutorial we will be able to use Mongoose model methods to query, create, update, and delete records in our database.
const mongoose = require('mongoose');
let todoSchema = new mongoose.Schema( { title: { type: String, required: true, }, complete: { type: Boolean, default: false, required: true, }, tags: [String], }, { timestamps: true });
module.exports = mongoose.model('Todo', todoSchema);MongoDB has built-in, albeit simple, validation. When defining a property on our object, we can specify if it is required. No instance of the model will be saved to the database if a required field is not satisfied. Additionally, we can set a default value to properties - in our case, a newly created todo starts with the property complete set as false.
Creating a todo
It is possible to run a complete Express server from a single file. Because we are modularizing our application, we will create a separate file for our routes and require it in server.js.
const todoRoutes = require('./api/todos/routes');// ...app.use(todoRoutes);The Router() method creates an instance of a router. A router object, in Express, is an isolated instance of middleware and routes. Middleware are functions that execute during the lifecycle of a request to the server.
router.METHOD() - where .METHOD() is one of the HTTP verbs (GET, POST, DELETE, PUT ) - accepts two arguments: 1) the path, and 2) a callback function.
Callback functions used on routing methods are referred to as route handlers. Route handlers are given the request and response objects as arguments. You can name these arguments whatever you want, but for convention you will very often see them written as req and res.
Below we've created our first route - a POST request to the path '/todos'. We've also created our first route handler.
const express = require('express');const router = express.Router();
router.post('/todos', async (req, res) => { try { console.log(req.body); } catch (err) { res.status(500).json({ Error: 'Internal Error' }); }});
module.exports = router;In Express, the req object represents the HTTP request. The body property on the req object contains the data we send to the server. In our route handler, let's test to see the contents of the req.body by logging it to the console.
We will be using cURL to test our API endpoints. cURL stands for "Client URL" and is a command-line tool for transferring data to a server.
From the terminal, using cURL, make a POST request to our API. We use the -d flag to pass data and -H to pass headers. For reference here is a cURL cheat sheet.
curl -X POST \-d '{"title": "Gym, Tan, Laundry", "tags": ["fitness", "grooming", "chores"]}' \-H "Content-Type: application/json" \http://localhost:1337/todosExpress does not automatically parse the request body for us so we get "undefined" in the console instead of the data we sent with our cURL command.
undefinedPOST /todos 200 1.089 ms - -To fix this we need to require the body-parser middleware. We need to place the bodyParser object before our routes, and because we are sending JSON data via cURL, we need to use the json() method that belongs to bodyParser.
const bodyParser = require('body-parser');
app.use(bodyParser.json());app.use(todoRoutes);Now let's try our cURL command again. If we have done everything correctly, we should see the req.body in the console.
POST /todos - - ms - -{ title: 'Gym, Tan, Laundry', tags: [ 'fitness', 'grooming', 'chores' ]}Now that Express is parsing req.body correctly, we can create a todo. First, we will modularize our app even further and separate our route handler into it's own file.
At the top of controller.js pull in the Todo model. As I mentioned previously, Mongoose models come with a set of methods for interacting with MongoDB.
In the createTodo() function we create a new instance of the Todo model and pass it the request body as it's argument. In MongoDB an instance of a model is called a document.
Then in our try/catch block we call save(), which is a Mongoose model instance method. If our document saves, we send a JSON response with the todo we just created to the client. If the save fails, we send a JSON error message.
const Todo = require('./model');
const createTodo = async (req, res) => { const todo = new Todo(req.body); try { await todo.save(); res.json(todo); } catch (err) { res.status(500).json({ Error: 'Internal Error' }); }};
module.exports = { createTodo,};Now let's refactor routes.js to require our controller methods.
const express = require('express');const router = express.Router();const controller = require('./controller');
router.post('/todos', controller.createTodo);
module.exports = router;From the terminal, run our cURL POST command again. The server should respond with our new todo object.
{ "title": "Gym, Tan, Laundry", "complete": false, "tags": [ "fitness", "grooming", "chores" ], "_id": "630e84f9689ed76663a7df66", "createdAt": "2022-08-30T21:45:29.928Z", "updatedAt": "2022-08-30T21:45:29.928Z", "__v": 0}For good measure, create a few more todos using our cURL command, as in our next controller method we will fetch all the todos in our database.
Fetching todos
In controller.js create a getTodos() function. Inside our try block, we call find() on our model and save the results to a variable. find() - a model method - retrieves a collection of documents, and accepts a filter object as it's argument. Because we want to retrieve all todos, at this time we will pass nothing to find().
const getTodos = async (req, res) => { try { const todos = await Todo.find(); res.json(todos); } catch (err) { res.status(500).json({ Error: 'Internal Error' }); }};
module.exports = { createTodo, getTodos,};Don't forget to create the route for which we pass our getTodos() function as a handler. This time we are using the get() method on our router. The path will remain '/todos'.
router.get('/todos', controller.getTodos);Now let's make a GET request via cURL in the terminal. If we've done our job correctly, we should see a JSON array of todo objects.
curl -sG http://localhost:1337/todos \| jq[ { "_id": "630e84f9689ed76663a7df66", "title": "Gym, Tan, Laundry", "complete": false, "tags": [ "fitness", "grooming", "chores" ], "createdAt": "2022-08-30T21:45:29.928Z", "updatedAt": "2022-08-30T21:45:29.928Z", "__v": 0 }, { "_id": "630e8596689ed76663a7df68", "title": "Gym", "complete": false, "tags": [ "fitness" ], "createdAt": "2022-08-30T21:48:06.235Z", "updatedAt": "2022-08-30T21:48:06.235Z", "__v": 0 }, { "_id": "630e8618689ed76663a7df6e", "title": "Tan", "complete": false, "tags": [ "grooming" ], "createdAt": "2022-08-30T21:50:16.612Z", "updatedAt": "2022-08-30T21:50:16.612Z", "__v": 0 }, { "_id": "630e8643689ed76663a7df72", "title": "Laundry", "complete": false, "tags": [ "chores" ], "createdAt": "2022-08-30T21:50:59.309Z", "updatedAt": "2022-08-30T21:50:59.309Z", "__v": 0 }, { "_id": "630e8663689ed76663a7df76", "title": "GTL", "complete": false, "tags": [ "fitness", "grooming", "chores" ], "createdAt": "2022-08-30T21:51:31.467Z", "updatedAt": "2022-08-30T21:51:31.467Z", "__v": 0 }]Get a single todo
In addition to req.body, the req object also has a params property. The params property is also an object and will contain the URL parameters if there are any. In our route below, Express will use regular expressions to match :id to any string that follows '/todos/' in our URL, and create a property on the params with id as the key and the string as it's value. We can name a param whatever we want, but for convention sake we are going with :id.
router.get('/todos/:id', controller.getTodo);Let's say we have a todo with the id of 1234. Ideally, the route we would use to fetch that particular todo would be http://localhost:1337/todos/1234. In getTodo() we could grab that id from the URL by accessing the params property on the req object. We then pass it to the findOne() model method. findOne() takes a filter object as it's argument, and in this case, we are looking for the todo with the id that matches the id in our req.params.
const getTodo = async (req, res) => { try { const todo = await Todo.findOne({ _id: req.params.id }); res.json(todo); } catch (err) { res.status(500).json({ Error: 'Internal Error' }); }};
module.exports = { createTodo, getTodo, getTodos,};If successful we send the todo to the client as JSON. If unsuccessful, we respond with an error message. Test it out with cURL.
curl -sG http://localhost:1337/todos/630e84f9689ed76663a7df66 \| jq{ "_id": "630e84f9689ed76663a7df66", "title": "Gym, Tan, Laundry", "complete": false, "tags": [ "fitness", "grooming", "chores" ], "createdAt": "2022-08-30T21:45:29.928Z", "updatedAt": "2022-08-30T21:45:29.928Z", "__v": 0}Update a todo
The path for updating a todo will be the same as retrieving a todo but the route will have the distinctions of sending a PUT request instead of a GET, and it's own handler: updateTodo().
router.put('/todos/:id', controller.updateTodo);Create updateTodo() in our controller.js file. findOneAndUpdate() is another model method that accepts a filter object, an object containing the fields we want to update, and an options object.
findOneAndUpdate() is atomic, meaning that it locates the document and updates it all at once. In contrast, when updating a document by calling save() on the instance, you must retrieve it first using findOne().
In findOneAndUpdate() we are retrieving the todo whose id matches the :id in the URL paramaters or req.params, passing in the req.body which has the data we want to update, and an options object. The new property returns the document after update (the default behaviour is to return the document pre-update), and the useFindAndModify property disables deprecation warnings in the console (don't worry this is recommended).
const updateTodo = async (req, res) => { try { const todo = await Todo.findOneAndUpdate({ _id: req.params.id }, req.body, { new: true, useFindAndModify: false, }); res.json(todo); } catch (err) { res.status(500).json({ Error: 'Internal Error' }); }};
module.exports = { createTodo, getTodo, getTodos, updateTodo,};Now let's send a put request using cURL. Let's mark this todo as complete by passing an object with complete: true as the field we want to update.
curl -X PUT \http://localhost:1337/todos/630e84f9689ed76663a7df66 \-H 'Content-Type: application/json' \-d '{"complete": true}' \| jq{ "_id": "630e84f9689ed76663a7df66", "title": "Gym, Tan, Laundry", "complete": true, "tags": [ "fitness", "grooming", "chores" ], "createdAt": "2022-08-30T21:45:29.928Z", "updatedAt": "2022-08-30T22:10:37.641Z", "__v": 0}Delete a todo
Much like updating a todo, we need an id to retrieve the particular todo from the database we want to delete. With that said, our DELETE request's path will need an :id parameter much like GET and PUT.
router.delete('/todos/:id', controller.deleteTodo);We pass the id from req.params to deleteOne()'s filter object. deleteOne() deletes the first document that matches the filter.
const deleteTodo = async (req, res) => { try { await Todo.deleteOne({ _id: req.params.id }); res.json({ Success: 'Successfully deleted todo.' }); } catch (err) { res.status(500).json({ Error: 'Internal Error' }); }};
module.exports = { createTodo, deleteTodo, getTodo, getTodos, updateTodo,};Once resolved, we send a success message to the client.
curl -X DELETE http://localhost:1337/posts/63046ce46440276676025579{"Success":"Successfully deleted todo."}CORS
So far we've been testing our API on the command-line with cURL. Let's see what happens if we try and log our data to the client, or more specifically, the browser console.
Create an index.html, and make a fetch() call to our server.
<!DOCTYPE html><html lang="en"> <head> <meta charset="UTF-8" /> <meta http-equiv="X-UA-Compatible" content="IE=edge" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> <title>Todo List API</title> </head> <body> <script> fetch('http://localhost:1337/posts') .then(response => response.json()) .then(data => console.log(data)); </script> </body></html>If we open the browser console we see we get an error.
Access to fetch at 'http://localhost:1337/todos' from origin 'http://127.0.0.1:5500' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
This is expected because "cross-domain" requests are blocked by default. This is a browser security feature to stop nefarious websites from stealing your data. If we were using Node to render our HTML, we would not have this issue as the HTML and server are of "same origin".
To fix this, we have to enable cross-origin resource sharing (CORS) in our application. Inside of our middleware file we will create a function that sets the response headers, and then calls next(). next() is function which simply tells Node to run the next middleware function in the sequence, which in our case is our route handlers.
const cors = (req, res, next) => { res.setHeader('Access-Control-Allow-Origin', '*'); res.setHeader('Access-Control-Allow-Methods', 'GET,PUT,POST,DELETE'); res.setHeader( 'Access-Control-Allow-Headers', 'Origin, X-Requested-With, Content-Type, Accept, Cache-Control' ); next();};Require our middleware in the server.js file.
const middleware = require('./lib/middleware');Order matters when we call our middleware. Our cors() function needs to precede our routes otherwise the response headers will not be set for each route call. Node's Express().use() function is a catch-all - it applies the middleware regardless of HTTP method.
app.use(middleware.cors);app.use(todosRoutes);Now if we refresh our index.html we should see our fetch call is working and we are successfully logging our Todos to the browser console.
(5) [{…}, {…}, {…}, {…}, {…}]
404 Not Found
Currently, if we tried to access a route on our API that did not exist, Node will return HTML. This is not ideal. If we want our API to be client agnostic we need to return JSON.
curl http://localhost:1337/todoss #typo
<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"><title>Error</title></head><body><pre>Cannot GET /todoss</pre></body></html>Create a function in middleware.js that accepts the req and res arguments, and returns a response with the 404 status and a JSON error message.
const notFound = (req, res) => res.status(404).json({ Error: 'Resource Not Found' });
module.exports = { notFound,};The order which we call our middleware is important. If no request matches any of our route handlers, our notFound() middleware will run. In server.js call notFound() after the call to our routes.
app.use(todoRoutes);app.use(middleware.notFound);Now if we try to access that non-existent resource again, we will get our JSON error instead of HTML.
curl http://localhost:1337/todoss{"Error":"Resource Not Found"}Cleaning up the controller
We have a bit of redundancy - there is a try/catch block in each one of our controller functions. The catch block specifically repeats the same JSON error message. A try/catch block is how we normally handle errors in async/await, but we can create a higher order function and apply it to each our route handlers to make our code more DRY.
A higher order function is a function that either accepts or returns another function. In our case, our catchWrapper() function does both. It accepts our route handler as it's argument and returns an async function that has the req, res and next arguments. Because we are returning an async function we can use a try/catch block inside it. Inside the try block we await the successful execution of our handler code, while passing it the req, res and nextarguments. If our handler fails, the catch block will pass the error to the next() function, which will call our handleError() middleware function.
module.exports = function (handler) { return async function (req, res, next) { try { await handler(req, res, next); } catch (err) { next(err); } };};Inside our middleware file, create the handleError() function. In Express, error handling functions have access to an error argument, which is the first argument. You must pass all four arguments (err, req, res, and next) in order to access it, otherwise Express will recognize the first argument as req.
const handleError = (err, req, res, next) => { console.error('Error:', err); res.status(500).json({ Error: 'Internal Error' });};
module.exports = { cors, handleError, notFound,};We never really want the end user to know what exactly is going on beneath the hood when there is an error. Exposing the end user to sever errors makes our code vulnerable to attack. This is why for the API user we simply send back a status of 500 and a generic message of "Internal Error", while for ourselves we log the actual error to the console.
Modify the routes file to use our catchWrapper() function to "wrap" each route handler.
const catchWrapper = require('../lib/catch-wrapper');
router.get('/todos', catchWrapper(controller.getTodos));router.post('/todos', catchWrapper(controller.createTodo));router.get('/todos/:id', catchWrapper(controller.getTodo));router.put('/todos/:id', catchWrapper(controller.updateTodo));router.delete('/todos/:id', catchWrapper(controller.deleteTodo));In our server.js place the handleError() middleware function after our routes but before notFound(). Remember how our controller functions are wrapped in a try/catch block inside of our catchWrapper() function? Remember how the catch block passes the error to next()? Well the next middleware in the sequence is handleError().
app.use(cors);app.use(todoRoutes);app.use(middleware.handleError);app.use(middleware.notFound);Now we can refactor our controller.js to be more succinct and without the repetitive try/catch blocks. If none of our controller functions resolve correctly, the error will be thrown to the handleError() middleware.
const getTodo = async (req, res) => { const todo = await Todo.findOne({ _id: req.params.id }); res.json(todo);};
const getTodos = async (req, res) => { const todos = await Todo.find(); res.json(todos);};
const createTodo = async (req, res) => { const todo = new Todo(req.body); await todo.save(); res.json(todo);};
const updateTodo = async (req, res) => { const todo = await Todo.findOneAndUpdate({ _id: req.params.id }, req.body, { new: true, useFindAndModify: false, }); res.json(todo);};
const deleteTodo = async (req, res) => { await Todo.deleteOne({ _id: req.params.id }); res.json({ Success: 'Successfully deleted todo!' });};Query strings
The req object has quite a few properties, with body and params being two that we looked at. req also has a query property, which is an object. req.query contains a property for each query string parameter that is in the route.
For the URL route http://localhost:1337/posts?page=1&limit=2&tag=fitness the req.query object woud look like:
{ "page": 1, "limit": 2, "tag": "fitness"}Like parameters, we can use query strings to send data with our database queries. Let's refactor getTodos() in controller.js to accept query strings and paginate our data.
const getTodos = async (req, res) => { const { limit = 0, page = 0, tag } = req.query;
let filter = {}; if (tag) filter.tags = tag;
const todos = await Todo.find(filter) .skip(page * limit) .limit(limit);
const totalCount = await Todo.countDocuments(filter);
res.json({ todos, perPage: todos.length, totalPages: Math.ceil(totalCount / limit), totalCount, });};Create variables for limit, page, and tag by destructuring the req.query object. Give limit and page default values of 0 just incase neither query string is passed, we show the complete list of todos starting from the beginning.
Previously, we passed zero arguments to our find() model method because we wanted to retrieve all todos. This time we programmatically build our filter starting with declaring an empty filter object. Next we check to see if the tag variable returns truthy - that is, it is not undefined, nor does it contain an empty string. If truthy, we set filter.tags to tag. In our todo schema, tags is defined as an array of strings. In Mongoose, searching an array field for a string value is as simple as writing a filter object like { tags: "fitness" }. If tag is falsy, the filter remains an empty object, which is as good as passing nothing to find().
Which todos our query produces is wholly dependent on the filter, and the limit and page values provided by the query string. We chain to the find() method two other methods: skip() and limit(). For skip() we pass in the value for page * limit, while for limit() we pass in limit.
For example, if we have 10 documents, and the value of limit is 2 and page is 2, then our find() query would return the next two todos (5, 6), starting from 5. Technically 5,6 are on the 3rd page, because the first page is actually 0. And the next page would consist of todos 7,8, and the next page after that 9,10.
countDocuments() is another model method which returns - you guessed it - the number of documents in a collection. countDocuments() also takes a filter object as an argument. The totalCount variable is the total number of documents with the filter applied, or in our case the total number of documents with a specific tag.
To round off our paginated collection of todos we return an object with the following properties: todos, perPage (the length of the todos array), totalPages (totalCount divided by the limit), and totalCount.
Let's test our query strings in cURL:
curl -sG http://localhost:1337/todos\?tag\='fitness' | jq{ "todos": [ { "_id": "630e84f9689ed76663a7df66", "title": "Gym, Tan, Laundry", "complete": true, "tags": [ "fitness", "grooming", "chores" ], "createdAt": "2022-08-30T21:45:29.928Z", "updatedAt": "2022-08-30T22:10:37.641Z", "__v": 0 }, { "_id": "630e8596689ed76663a7df68", "title": "Gym", "complete": false, "tags": [ "fitness" ], "createdAt": "2022-08-30T21:48:06.235Z", "updatedAt": "2022-08-30T21:48:06.235Z", "__v": 0 }, { "_id": "630e8d017794f16ff10a53fd", "title": "GTL", "complete": false, "tags": [ "fitness", "grooming", "chores" ], "createdAt": "2022-08-30T22:19:45.611Z", "updatedAt": "2022-08-30T22:19:45.611Z", "__v": 0 } ], "perPage": 3, "totalPages": null, "totalCount": 3}Now with all three query string parameters:
curl -sG http://localhost:1337/todos\?tag\='fitness'\&limit\=2\&page\=0 | jq{ "todos": [ { "_id": "630e84f9689ed76663a7df66", "title": "Gym, Tan, Laundry", "complete": true, "tags": [ "fitness", "grooming", "chores" ], "createdAt": "2022-08-30T21:45:29.928Z", "updatedAt": "2022-08-30T22:10:37.641Z", "__v": 0 }, { "_id": "630e8596689ed76663a7df68", "title": "Gym", "complete": false, "tags": [ "fitness" ], "createdAt": "2022-08-30T21:48:06.235Z", "updatedAt": "2022-08-30T21:48:06.235Z", "__v": 0 } ], "perPage": 2, "totalPages": 2, "totalCount": 3}There you have it, a Node/Express REST API that is client agnostic. A while back I simulated a fake REST API using JSON-Server and React. We could replace JSON-Server in that project with this Express server and would have to change very little code to get it working.
It is still a basic API, as we have not explored relationships like has many, belongs to, and we've barely scratched the surface as far as Express or Mongoose is concerned.