Web maps: the Mer et Demeures project

Since ancient times, maps have always allowed to seek and find information in “spatial mode”. Today it is very common for them to be inserted into websites, as we did in the case of Mer et Demeures.

Talk transcripts (2 part series)

Full-text search in Django with PostgreSQL
Web maps: the Mer et Demeures project

Contents

What is a web map?
GeoDjango
PostGIS
Leaflet JS
A basic map example
The Mer et Demeures project
Conclusions

What is a web map?

A web map is the representation of a geographical system, providing spatial data and information.

The map can have different features, depending on what is needed:

it can be static or dynamic;
may be interactive or not;
can be designed using photographic images or vector models.

In general, in order to create one, we need a spatial database for data-storage and a Javascript library to design it on the webpage.

We used 3 fundamental software components for Mer et Demeures: GeoDjango, PostGIS and Leaflet JS. Let’s see them one at a time.

GeoDjango

This one is not an external package, on the contrary it has been part of Django, as a contrib package, since version 1.0: it turns Django, as we were saying, into a geographic spatial framework.

How does it work?

First of all, GeoDjango provides us with spatial field types - there is quite a few of them - and allows us to perform spatial queries directly in Django’s ORM. Also, the fields are already integrated into the admin.

GeoDjango currently supports 4 geospatial databases.

PostGIS

We used PostGIS in this project: we opted for it because we were already using PostgreSQL, but also because it represents GeoDjango’s backend with the most extensive support.

This is a PostgreSQL extension which directly integrates spatial data.

Alongside with GeoDjango, PostGIS provides us with specific data types so as to store our spatial information. In addition, there are indexes optimized for spatial data, as well as specific functions that we can use to search for this kind of data.

Leaflet JS

We added Leaflet to our stack, for the frontend map, for a number of reasons.

Firstly, it is one of the most used Javascript libraries for rendering maps on a browser.

Another aspect of primary importance, both for the client and for us developers, consists in the fact that it is a Free Software, with a large community of developers participating in its development. Moreover, it is mobile friendly, which is essential in the present era, and it is also very light.

Plus, it is extremely easy to use: the documentation is accessible and the result you get has excellent performance.

A basic map example

Let’s take Django’s Blog application, which can be found in the documentation, and test it with these three models.

from django.db import models


class Blog(models.Model):
    name = models.CharField(
        max_length=100
    )


class Author(models.Model):
    name = models.CharField(
        max_length=200
    )


class Entry(models.Model):
    blog = models.ForeignKey(
        Blog, on_delete=models.CASCADE
    )
    headline = models.CharField(
        max_length=255
    )
    authors = models.ManyToManyField(
        Author
    )

We are going to add spatial data to this application and render them in a web map. Starting from this example, the first step is to change the settings:

INSTALLED_APPS = [
    # …
    "django.contrib.gis",
]

DATABASES = {
    "default": {
        "ENGINE": "django.contrib.gis.db.backends.postgis",
        # …
    }
}

We should also activate the PostGIS extension with a migration:

from django.contrib.postgres.operations import (
    CreateExtension,
)
from django.db import migrations


class Migration(migrations.Migration):
    dependencies = [
        ("blog", "0001_initial")
    ]
    operations = [
        CreateExtension("postgis")
    ]

The other change we need to make is adding a PointField, importing it from GeoDjango. We have also added a property, which will then be useful in Leaflet, so as to render longitude and latitude.

from django.contrib.gis.db.models import (
    PointField,
)
from django.db import models


class Entry(models.Model):
    # …
    point = PointField()

    @property
    def lat_lng(self):
        return list(
            getattr(
                self.point, "coords", []
            )[::-1]
        )

The easiest way to change our entries consists in changing Admin in order to use a specific one. We can also indicate distinct attributes and zoom by default.

This will be the final result:

We can zoom, move around the map and interact with it: all with a few lines of code.

Once we have edited the entries and added points to the admin, we can show them on a real map. We use views and urls to do this:

from django.urls import path
from django.views.generic import (
    ListView,
)
from .models import Entry


class EntryList(ListView):
    queryset = Entry.objects.filter(
        point__isnull=False
    )


urlpatterns = [
    path("map/", EntryList.as_view()),
]

After defining the EntryList, we map it to a url: and here is where the backend work ends. The following step concerns writing the template.

We start by including JS Leaflet’s style and the JavaScript library in the head. Then we write a div which will allow us to render the map in the html part:

<script type="text/javascript">
  var m = L.map('m').setView(
    [43.77, 11.26], 15
  );
  L.tyleLayer(
    '//{s}.tile.osm.org/{z}/{x}/{y}.png'
  ).addTo(m);
  {% for e in object_list %}
    L.marker({{e.lat_lng}}).addTo(m);
  {% endfor %}
</script>

As to what concerns the Javascript part: L stands for Leaflet, we render the map in our Div and set the map view, latitude, longitude and zoom. We also choose the tile which needs to be inserted: we use Open Street Map for the graphic part in this case, otherwise we would not be able to see anything.

We then insert a loop with the list of markers and we ask Leaflet to print one with latitude and longitude for each marker.

WARNING: PostGIS first asks for longitude and then latitude, while Leaflet, on the contrary, first asks for latitude and then longitude. That is why we have created a property.

And here is the final result:

This code is actually working: you can try using it too, by following all the steps!

The Mer et Demeures project

Compared to what we have just seen, the situation we have faced in this project is much more complex and requires a more detailed approach.

Mer et Demeures is a French company, based in Provence, selling and renting properties by the sea all over the world. Being active since 2014, it has a portal, which has been translated into 8 languages, with over 100,000 ads in 40 countries and 6 continents.

This massive project had already been in production for a long time and required a very complex and interactive map.

Here is a screenshot of the mobile 1.0 version, created with Django 1.5, Python 2.7, PostgreSQL 9.3, with just one text field for spatial data storage. Leaflet had already been used, but only for a static view of the map.

The result of our intervention is a very interactive map.

We used:

Django 2.2 with the active GeoDjango module activated
Python 3.6
PostegreSQL 10
PostGIS 2.4 to store spatial data
Leaflet 1.4

To understand the difference in comparison with the basic examples above, here is an excerpt of the models.

For example, we used the multipolygon to display city boundaries, the ad model with a PointField and a much more complex hierarchy.

from django.db import models
from django.contrib.gis.db.models import (
    MultiPolygonField,
    PointField,
)


class City(models.Model):
    borders = MultiPolygonField()


class Ad(models.Model):
    city = models.ForeignKey(
        City, on_delete=models.CASCADE
    )
    location = PointField()

We couldn’t use a template for this project, but we decided to implement a RESTful API and provide it to the frontend.

$ pip install djangorestframework     # RESTful API
$ pip install djangorestframework-gis # Geographic add-on
$ pip install django-filter           # Filtering support

INSTALLED_APPS = [
    # …
    "django.contrib.gis",
    "rest_framework",
    "rest_framework_gis",
    "django_filters",
]

The first step for implementing an API is to write a model serializer: we inherited a basic one, in which we specified model, field and additional fields.

from rest_framework_gis.serializers import (
    GeoFeatureModelSerializer,
)
from .models import Ad


class AdSerializer(
    GeoFeatureModelSerializer
):
    class Meta:
        model = Ad
        geo_field = "location"
        fields = ("id",)

The next step consisted in importing a view:

from rest_framework.viewsets import (
    ReadOnlyModelViewSet,
)
from rest_framework_gis.filters import (
    InBBoxFilter,
)
from .models import Ad
from .serializers import AdSerializer


class AdViewSet(ReadOnlyModelViewSet):
    bbox_filter_field = "location"
    filter_backends = (InBBoxFilter,)
    queryset = Ad.objects.filter(
        location_isnull=False
    )
    serializer_class = AdSerializer

At this point we use DefaultRouter and shut the backend work.

from rest_framework.routers import (
    DefaultRouter,
)
from .views import AdViewSet

router = DefaultRouter()
router.register(
    r"markers",
    AdViewSet,
    basename="marker",
)
urlpatterns = router.urls

The result is a GeoJSON, a standard way of providing geo-spatial data:

{
  "type": "FeatureCollection",
  "features": [
    {
      "id": 1,
      "type": "Feature",
      "geometry": {
        "type": "Point",
        "coordinates": [
          11.255814, 43.769562
        ]
      },
      "properties": {}
    }
  ]
}

Here there are no additional properties, but there were several in the original project, so as to render pop-ups with additional data (such as currency, price…).

At this point the ball is in the frontend’s court, which will be able to query these data.

Let’s make a first example with JavaScript:

<script type="text/javascript">
  var m = L.map("m").setView(
    [43.77, 11.26],
    15
  );
  L.tileLayer(
    "//{s}.tile.osm.org/{z}/{x}/{y}.png"
  ).addTo(m);
  fetch("/markers").then(function (
    results
  ) {
    L.geoJSON(results).addTo(m);
  });
</script>

The first two lines render the map.
The last ones are necessary for rendering the entire GeoJSON.

The main difference, compared to the previous procedure, is that the GeoJSON can contain polygons, markers, basically anything that respects its standards: this way we are able to render everything in one shot.

Leaflet also provides events: we were interested in Moveend, to be invoked when the centre of the map is moved and when animations end. This is a very useful aspect in case of strong interaction with the map, as for Mer et Demeures.

As for what concerns the React part, we used a React Leaflet, which is not a separate library but only a station (that is why Leaflet needs to be installed too):

import React from "react";
import {
  Map,
  TileLayer,
  GeoJSOn,
} from "react - leaflet";

export default class Map extends Component {
  state = {
    geoJson: {},
  };
  onMove = () => {
    fetch("/markers").then((geoJson) =>
      this.setState({
        geoJson,
      })
    );
  };
  // render ( ) { ... }
}

We import React and the 3 components provided by the library and we initialize the state of our component in a new GeoJSON object, where all the new contents will be placed. Then there is the function that is invoked at the time of interaction.

Let’s see the Render method, which is very similar to the other:

render() {
    return (
        <Map center= {c} zoom= {z} onMoveend= {this.onMove}>
          <TileLayer url="//{s}.tile.osm.org/ {z}/{x}/{y}.png" />
          <GeoJSON data= {this.state.geoJson} />
        </Map>
    )
}

In our case we really had many ads to place and it is not recommended to render more than 100 markers on Leaflet because it slows it down a lot.

The solution we found consisted in clustering these markers on the backend side which expanded when the user clicked: on React this thing severely slowed down rendering.

Conclusions

In conclusion, these are the highlights of our work:

Thanks to Django’s internal stack, with GeoDjango and PostgreSQL, we found an immediate solution, without resorting to external services.
We made spatial and relational queries at the same time, which is very useful for having a good reactivity.
Simplicity allowed us to focus on highly advanced solutions, directly programmed by us, such as backend clustering, which permitted us to group points in queries without making advanced cash in global visualisation.
We used borders, limits, administrative boundaries.
We created dynamic spatial entities.

A special thanks to Mer et Demeures who allowed us to study and explore such an interesting topic, thanks to their project!

Content licensed under a Creative Commons Attribution-ShareAlike 4.0 International.
Paolo Melchiorre, Carmelo Catalfamo
Originally posted on 20tab.com