VueX Modulator - Part 2: dispatching incoming data

This article follows [article:vuex-modulator-part-1-generating-vuex-modules]. It's a step by step explanation on how to manage data coming from an API in a VueX store.

The modulator was a good thing, creating module and handling data became easy. Until the day an API responded with this:

[
  {
    "id": 1,
    "name": "John Doe",
    "posts": [
      {
        "id": 1,
        "title": "A great post about data consistency",
        "content": "<some uninteresting content>",
        "published_at": "..."
      }
    ]
  }
]

Well… Getting the users got me his/her posts. Storing them in VueX is a great idea (if the list is complete, of course), we’ll save one request !

But that breaks the way the modulator works, because changing the load<Type> generic action to maybe handle posts is not what we want. A good idea would be to declare somewhere the relations between data types, and update the modulator to dispatch them in the right VueX module.

Let’s do this !

Before we continue here, I assume you already use VueX and you already have read the previous post:

[article-card:vuex-modulator-part-1-generating-vuex-modules]

A repo exists if you want to check the file structure the commits, etc:

[github-repo-card:el-cms/example-vuex-module-generator]

Declare the relations between data types

That’s a bit like creating models for some framework ORM, without the fields. In my first attempts, I wanted to add field validation in these “models”, but stopped, lack of time.

For the example, let’s say we have those data types:

Users: have many articles, many tasks, but have one billing address and one delivery address
Addresses: each address belongs to one user, as billing adress or delivery adress
Articles: belongs to one user, have many tags
Tags: have many articles
Tasks: belongs to one user

The notation of “has one, “has many”, “belongs to” are only useful for the formulation, here. We have to check the data we receive to put things in shape:

Users:

{
  "id": 1,
  "name": "Freddy Krueger",
  "billing_address_id": 1,
  "delivery_address_id": 2
}

Addresses:

{
  "id": 1,
  "number": "1428",
  "street": "Elm Street"
}

Articles:

{
  "id": 1,
  "title": "How to melt staircases",
  "user_id": 1,
  "tag_ids": [1, 2, 3]
}

Tags:

{
  "id": 1,
  "name": "funny",
  "post_ids": [1, 8, 4]
}

Tasks:

{
  "id": 1,
  "title": "Do something",
  "done": false,
  "user_id": 1
}

Depending on what you get from the API, you may have only the ids, or the complete records. In the last case, the field name will probably be user or tags instead of user_id and tag_ids.

The user data type have two adresses, with different names, but both pointing to the address data type

I came to define relations between data types like this:

const entityTypes = {
  user: {
    singular: 'user',
    plural: 'users',
    relations: {
      many: ['articles', 'tasks'],
      one: [
        {field: 'billing_address', type: 'address'},
        {field: 'delivery_address', type: 'address'},
      ],
      habtm: [],
    },
  },
  address: {
    singular: 'address',
    plural: 'addresses',
    relations: {
      many: [],
      one: ['user'],
      habtm: [],
    },
  },
  article: {
    singular: 'article',
    plural: 'articles',
    relations: {
      many: [],
      one: ['user'],
      habtm: [
        {name: 'tags', field: 'article_id', assoc: 'tag_id'},
      ],
    },
  },
  tag: {
    singular: 'tag',
    plural: 'tags',
    relations: {
      many: [],
      one: [],
      habtm: [
        {name: 'articles', field: 'tag_id', assoc: 'article_id'},
      ],
    },
  },
  task: {
    singular: 'task',
    plural: 'tasks',
    relations: {
      many: [],
      one: ['user'],
      habtm: [],
    },
  },
}

The singular and plural keys are here to easily find where data comes from and where it should go.

In user, the relations for the addresses should point to the same type, so we specify the type and its field name. It should work for many relations (i.e.: one user has many published posts and many drafts; both relations are articles).

The habtm relations may seems tricky:

name is the related type
field is the name of the current type reference
assoc is the name of the associated type reference

Create the code to do something useful

The hard part here is to find the correct type for a given key in an incoming object. To achieve this, we should have methods to quickly jump from plurals to singulars and vice-versa, methods to analyse variables types,

Methods, methods everywhere

In a file named utils, we add all the methods that should be used somewhere else:

// src/utils.js

export default {
  // Return true if the given value is an object
  isObject (object) { return object !== null && typeof object === 'object'},

  // Return true if the given object has the given key
  objectHasKey (object, property) {
    if (this.isObject(object)) { return Object.prototype.hasOwnProperty.call(object, property) }
    return false
  },
}

In a file named data_types/types.js, we put our data types. If you have many datatypes (5 or more), you may want to split this file into multiple files in a sub folder. For our example, let’s keep it simple (commit #986504e6):

// data_types/types.js
export default {
  user: { /* copy/paste the definition from previous section */ },
  address:  { /* copy/paste the definition from previous section */ },
  post:  { /* copy/paste the definition from previous section */ },
  tag:  { /* copy/paste the definition from previous section */ },
  task:  { /* copy/paste the definition from previous section */ },
}

In a file named data_types/index.js we put all the methods needed to use this:

// data_types/index.js

import utils from '../utils'
import dataTypes from './data_types'

// Creates an object like
// {
//   singulars: {
//     'users': 'user'
//   },
//   plurals: {
//     'user': 'users'
//   },
// }
const collectSingularsAndPlurals = function (dataTypes) {
  const singulars = {}
  const plurals = {}

  for (let key in dataTypes) {
    let type= dataTypes[key]
    if (utils.isObject(type)) {
      singulars[type.plural] = type.singular
      plurals[type.singular] = type.plural
    }
  }
  return {singulars, plurals}
}

// Find all plurals and singulars from data types
const pluralsAndSingulars = collectSingularsAndPlurals(dataTypes)

// Returns a singular name or the string without the last character
const singular = function (plural) {
  if (pluralsAndSingulars.singulars[plural]) { return pluralsAndSingulars.singulars[plural] }
  console.error(`Singular not found for ${plural}, falling back.`)
  return plural.slice(0, plural.length - 1)
}

// Returns a plural name or the string with an ending 's"
const plural = function (singular) {
  if (pluralsAndSingulars.plurals[singular]) { return pluralsAndSingulars.plurals[singular] }
  console.error(`Plural not found for ${singular}, falling back`)
  return `${singular}s`
}

Some of the methods here are not used in the code of this article, but are in the final files in the repository. I still put them here to explain the thought process.

You want some spaghettis with all that sauce?

When data comes from the API, it should be analyzed and dispatched in the corresponding modules. Happily for us, Vuex allows to access dispatch and commit in the actions methods.

If you forgot about dispatch and commit, dispatch calls another action, while commit executes a mutation. We’ll use dispatch, and I’ll explain it in the “Integrate with the modulator” section.

// Take an entity and dispatches the related data in corresponding modules
export default function (entity, dispatch, entityType) {
  dispatchOneRelations(entity, dispatch, entityType)
  dispatchManyRelations(entity, dispatch, entityType)
  dispatchHABTMRelations(entity, dispatch, entityType)

  /* --> Here, we should save the entity in the store <-- */
}

Let’s write the code for all those methods in data_types/index.js (commit #b7d61af):

// data_types/index.js

// ... previous code

const dispatchOneRelations = function (entity, dispatch, entityType) {
  const relations = dataTypes[entityType].relations
  let fieldName = ''
  let typeName = ''
  let foreignKey = ''
  for (const relation of relations.one) {
    // Is the relation defined as string or object ?
    if (utils.isObject(relation)) {
      fieldName = relation.field
      typeName = relation.type
      foreignKey = `${relation.field}_id`
    } else {
      fieldName = relation
      typeName = relation
      foreignKey = `${relation}_id`
    }

    // Check if the relation is present in the entity
    if (
      utils.objectHasKey(entity, fieldName) &&
      // Prevent dispatching entities with no Ids
      // (i.e.: an entity like { name: 'my tag' } with nothing else
      utils.objectHasKey(entity[fieldName], 'id') &&
      entity[fieldName].id
    ) {
      // Check if the foreign_key exists and set it if necessary
      if(!utils.objectHasKey(entity, foreignKey)){
        entity[foreignKey] = entity[fieldName].id
      }
      /*
      --> Here, we should save the related entity in the store <--
       */
    }

    // Anyway, we delete the key in the parent
    delete entity[fieldName]
  }

  return entity
}

const dispatchManyRelations = function (entity, dispatch, entityType) {
  // Boring code, check the repo if you're really interested
}

const dispatchHABTMRelations = function (entity, dispatch, entityType) {
  // Boring code, check the repo if you're really interested
}

// Take an entity and dispatches the related data in corresponding modules
// Between us, let's call it findAndDispatchEntities
export default function (entity, dispatch, entityType) {
  return new Promise((resolve, reject) => {
    let newEntity
    newEntity = dispatchOneRelations(entity, dispatch, entityType)
    newEntity = dispatchManyRelations(newEntity, dispatch, entityType)
    newEntity = dispatchHABTMRelations(newEntity, dispatch, entityType)

    resolve(newEntity)
  })
}

In the last function, we return a promise: we can handle dispatch errors directly in the Modulator.

It’s time to integrate this in our module generator.

Usage in the modulator

We’re setting up something that analyze entities to find other entities in them. The found entities may also have related data in them, so I propose this:

execute an API call in a store action
receive some data
call another store action to dispatch the related entities (let’s call it “dispatchAndCommit()")
dispatchAndCommit<Type> will use findAndDispatchEntities
findAndDispatchEntities will find related entities and call dispatchAndCommit<Type>() with the entities found.
The circle is complete, our store should be populated

First, create the dispatchAndCommit<Type> method:

// src/modulator/index.js

actions[`dispatchAndCommit${camelSingular}`] = ({commit, dispatch}, entity) => {
  findAndDispatchEntities(entity, dispatch, singular)
    .then((cleanEntity) => {
      // Commit the cleaned entity: all the fields where removed by findAndDispatchEntities
      commit(`set_${singular}`, cleanEntity)
    })
    .catch((error) => {console.log('An error occurred while processing data')})
}

For every module, a dispatchAndCommit<Type> method is now available. Let’s call it when we receive data:

actions[`load${camelPlural}`] = ({dispatch}) => {
  api.get(endpoint)
    .then((data) => {
      for (let i = 0; i < data.length; i++) {
        dispatch(`dispatchAndCommit${camelSingular}`, data[i])
      }
    })
    .catch((error) => {console.log(`ERROR in load${camelPlural}:`, error)})
}

You may update load<Singular>, create<singular> and update<singular> as well.

Query the store

The last interesting part, is to add some getters, to let’s say, get all the users posts.

Remember, we store all incoming data directly in the module’s state, with the entity id as a key. To filter the module’s state, we need a method which take the object as first argument, and a callback test function; then iterate on the object’s keys.

// src/utils.js

export default {
  // ...

  // if "first" argument is true, return the first object found and return it
  filterObject (obj, test, first = false) {
    if (obj !== null && typeof obj === 'object') {
      const results = {}
      for (const key in obj) {
        if (obj.hasOwnProperty(key)) {
          if (test(obj[key], key)) {
            if (first) {
              return obj[key]
            }
            results[key] = obj[key]
          }
        }
      }
      return results
    }

    throw new Error('The thing you try to filter is not an object.')
  },
}

And now you can have getters like this:

getters[`${lowCamelSingular}Related`] = (state, rootState) => (type, id) => {
  return utils.filterObject(rootState[type], (entity) => entity[`${singular}_id`] === id) || undefined
}

See commit #8b3f4f7 for the modulator changes.

Fake an api for quick tests

In the repo, I created a fake api module to return data to the modulator. If you start the project you can play with it, changing the order of data loading in App.vue, created() method (commit #86230553).

The results will be displayed in the view.

Going deeper

You may want to adapt the data types definitions to add the API endpoint, whether or not you want the type to be editable/destroyable and setup additionnal stuff like custom mutators/actions/getters in it, and use these definitions to generate your store… That’s easy to set up and makes everything easier to maintain.

Conclusion

This was one of the heaviest VueX adaptation i’ve made; setting up something like this takes a bit of time, but it’s worth it if you need the feature: it helps to reduce the number of api calls and manages a large part of the store for you.

That said, depending on the API you work with and the type of application, it may be of no use.

Note: The code in the repository is somewhat simplified, if you want to set up something like this, you may want to be more careful about what is done on what: check every key you manipulate, create objects generators to have the good structures of data, etc… You may be interested by those files I wrote in an attempt to get all my tools packaged.

You can leave a comment here if you wish, or find me on Gitter