REST Pagination in Spring

1. Overview

This tutorial will focus on the implementation of pagination in a REST API, using Spring MVC and Spring Data.

2. Page as Resource vs Page as Representation

The first question when designing pagination in the context of a RESTful architecture is whether to consider the page an actual Resource or just a Representation of Resources.

Treating the page itself as a resource introduces a host of problems such as no longer being able to uniquely identify resources between calls. This, coupled with the fact that, in the persistence layer, the page is not a proper entity but a holder that is constructed when necessary, makes the choice straightforward: the page is part of the representation.

The next question in the pagination design in the context of REST is where to include the paging information:

• in the URI path: /foo/page/1
• the URI query: /foo?page=1

Keeping in mind that a page is not a Resource, encoding the page information in the URI is no longer an option.

We’re going to use the standard way of solving this problem by encoding the paging information in a URI query.

3. The Controller

Now, for the implementation – the Spring MVC Controller for pagination is straightforward:

@GetMapping(params = { "page", "size" })
public List<Foo> findPaginated(@RequestParam("page") int page, 
  @RequestParam("size") int size, UriComponentsBuilder uriBuilder,
  HttpServletResponse response) {
    Page<Foo> resultPage = service.findPaginated(page, size);
    if (page > resultPage.getTotalPages()) {
        throw new MyResourceNotFoundException();
    }
    eventPublisher.publishEvent(new PaginatedResultsRetrievedEvent<Foo>(
      Foo.class, uriBuilder, response, page, resultPage.getTotalPages(), size));

    return resultPage.getContent();
}

In this example, we’re injecting the two query parameters, size and page, in the Controller method via @RequestParam.

Alternatively, we could have used a Pageable object, which maps the page, size, and sort parameters automatically. In addition, the PagingAndSortingRepository entity provides out-of-the-box methods that support using the Pageable as a parameter as well.

We’re also injecting both the Http Response and the UriComponentsBuilder to help with Discoverability – which we’re decoupling via a custom event. If that’s not a goal of the API, you can simply remove the custom event.

Finally – note that the focus of this article is only the REST and the web layer – to go deeper into the data access part of pagination you can check out this article about Pagination with Spring Data.

4. Discoverability for REST Pagination

Within the scope of pagination, satisfying the HATEOAS constraint of REST means enabling the client of the API to discover the next and previous pages based on the current page in the navigation. For this purpose, we’re going to use the Link HTTP header, coupled with the “next“, “prev“, “first” and “last” link relation types.

In REST, Discoverability is a cross-cutting concern, applicable not only to specific operations but to types of operations. For example, each time a Resource is created, the URI of that Resource should be discoverable by the client. Since this requirement is relevant for the creation of ANY Resource, we’ll handle it separately.

We’ll decouple these concerns using events, as we discussed in the previous article focusing on Discoverability of a REST Service. In the case of pagination, the event – PaginatedResultsRetrievedEvent – is fired in the controller layer. Then we’ll implement discoverability with a custom listener for this event.

In short, the listener will check if the navigation allows for a next, previous, first and last pages. If it does – it will add the relevant URIs to the response as a ‘Link’ HTTP Header.

Let’s go step by step now. The UriComponentsBuilder passed from the controller contains only the base URL (the host, the port and the context path). Therefore, we’ll have to add the remaining sections:

void addLinkHeaderOnPagedResourceRetrieval(
 UriComponentsBuilder uriBuilder, HttpServletResponse response,
 Class clazz, int page, int totalPages, int size ){

   String resourceName = clazz.getSimpleName().toString().toLowerCase();
   uriBuilder.path( "/admin/" + resourceName );

    // ...
   
}

Next, we’ll use a StringJoiner to concatenate each link. We’ll use the uriBuilder to generate the URIs. Let’s see how we’d proceed with the link to the next page:

StringJoiner linkHeader = new StringJoiner(", ");
if (hasNextPage(page, totalPages)){
    String uriForNextPage = constructNextPageUri(uriBuilder, page, size);
    linkHeader.add(createLinkHeader(uriForNextPage, "next"));
}

Let’s have a look at the logic of the constructNextPageUri method:

String constructNextPageUri(UriComponentsBuilder uriBuilder, int page, int size) {
    return uriBuilder.replaceQueryParam(PAGE, page + 1)
      .replaceQueryParam("size", size)
      .build()
      .encode()
      .toUriString();
}

We’ll proceed similarly for the rest of the URIs that we want to include.

Finally, we’ll add the output as a response header:

response.addHeader("Link", linkHeader.toString());

5. Test Driving Pagination

Both the main logic of pagination and discoverability are covered by small, focused integration tests. As in the previous article, we’ll use the REST-assured library to consume the REST service and to verify the results.

These are a few examples of pagination integration tests; for a full test suite, check out the GitHub project (link at the end of the article):

@Test
public void whenResourcesAreRetrievedPaged_then200IsReceived(){
    Response response = RestAssured.get(paths.getFooURL() + "?page=0&size=2");

    assertThat(response.getStatusCode(), is(200));
}
@Test
public void whenPageOfResourcesAreRetrievedOutOfBounds_then404IsReceived(){
    String url = getFooURL() + "?page=" + randomNumeric(5) + "&size=2";
    Response response = RestAssured.get.get(url);

    assertThat(response.getStatusCode(), is(404));
}
@Test
public void givenResourcesExist_whenFirstPageIsRetrieved_thenPageContainsResources(){
   createResource();
   Response response = RestAssured.get(paths.getFooURL() + "?page=0&size=2");

   assertFalse(response.body().as(List.class).isEmpty());
}

6. Test Driving Pagination Discoverability

Testing that pagination is discoverable by a client is relatively straightforward, although there is a lot of ground to cover.

The tests will focus on the position of the current page in navigation and the different URIs that should be discoverable from each position:

@Test
public void whenFirstPageOfResourcesAreRetrieved_thenSecondPageIsNext(){
   Response response = RestAssured.get(getFooURL()+"?page=0&size=2");

   String uriToNextPage = extractURIByRel(response.getHeader("Link"), "next");
   assertEquals(getFooURL()+"?page=1&size=2", uriToNextPage);
}
@Test
public void whenFirstPageOfResourcesAreRetrieved_thenNoPreviousPage(){
   Response response = RestAssured.get(getFooURL()+"?page=0&size=2");

   String uriToPrevPage = extractURIByRel(response.getHeader("Link"), "prev");
   assertNull(uriToPrevPage );
}
@Test
public void whenSecondPageOfResourcesAreRetrieved_thenFirstPageIsPrevious(){
   Response response = RestAssured.get(getFooURL()+"?page=1&size=2");

   String uriToPrevPage = extractURIByRel(response.getHeader("Link"), "prev");
   assertEquals(getFooURL()+"?page=0&size=2", uriToPrevPage);
}
@Test
public void whenLastPageOfResourcesIsRetrieved_thenNoNextPageIsDiscoverable(){
   Response first = RestAssured.get(getFooURL()+"?page=0&size=2");
   String uriToLastPage = extractURIByRel(first.getHeader("Link"), "last");

   Response response = RestAssured.get(uriToLastPage);

   String uriToNextPage = extractURIByRel(response.getHeader("Link"), "next");
   assertNull(uriToNextPage);
}

Note that the full low-level code for extractURIByRel – responsible for extracting the URIs by rel relation is here:

public static String extractURIByRel(final String linkHeader, final String rel) {
    if (linkHeader == null) {
        return null;
    }

    String uriWithSpecifiedRel = null;
    final String[] links = linkHeader.split(", ");
    String linkRelation = null;
    for (final String link : links) {
        final int positionOfSeparator = link.indexOf(';');
        linkRelation = link.substring(positionOfSeparator + 1, link.length()).trim();
        if (extractTypeOfRelation(linkRelation).equals(rel)) {
            uriWithSpecifiedRel = link.substring(1, positionOfSeparator - 1);
            break;
        }
    }

    return uriWithSpecifiedRel;
}

static Object extractTypeOfRelation(final String linkRelation) {
    final int positionOfEquals = linkRelation.indexOf('=');
    return linkRelation.substring(positionOfEquals + 2, linkRelation.length() - 1).trim();
}

7. Getting All Resources

On the same topic of pagination and discoverability, the choice must be made if a client is allowed to retrieve all the Resources in the system at once, or if the client must ask for them paginated.

If the choice is made that the client cannot retrieve all Resources with a single request, and pagination is not optional but required, then several options are available for the response to a get all request. One option is to return a 404 (Not Found) and use the Link header to make the first page discoverable:

Link=<http://localhost:8080/rest/api/admin/foo?page=0&size=2>; rel=”first”, <http://localhost:8080/rest/api/admin/foo?page=103&size=2>; rel=”last”

Another option is to return redirect – 303 (See Other) – to the first page. A more conservative route would be to simply return to the client a 405 (Method Not Allowed) for the GET request.

8. REST Paging With Range HTTP Headers

A relatively different way of implementing pagination is to work with the HTTP Range headers – Range, Content-Range, If-Range, Accept-Ranges – and HTTP status codes – 206 (Partial Content), 413 (Request Entity Too Large), 416 (Requested Range Not Satisfiable).

One view on this approach is that the HTTP Range extensions were not intended for pagination and that they should be managed by the Server, not by the Application. Implementing pagination based on the HTTP Range header extensions is nevertheless technically possible, although not nearly as common as the implementation discussed in this article.

9. Spring Data REST Pagination

In Spring Data, if we need to return a few results from the complete data set, we can use any Pageable repository method, as it will always return a Page. The results will be returned based on the page number, page size, and sorting direction.

Spring Data REST automatically recognizes URL parameters like page, size, sort etc.

To use paging methods of any repository we need to extend PagingAndSortingRepository:

public interface SubjectRepository extends PagingAndSortingRepository<Subject, Long>{}

If we call http://localhost:8080/subjects Spring automatically adds the page, size, sort parameters suggestions with the API:

"_links" : {
  "self" : {
    "href" : "http://localhost:8080/subjects{?page,size,sort}",
    "templated" : true
  }
}

By default, the page size is 20 but we can change it by calling something like http://localhost:8080/subjects?page=10.

If we want to implement paging into our own custom repository API we need to pass an additional Pageable parameter and make sure that API returns a Page:

@RestResource(path = "nameContains")
public Page<Subject> findByNameContaining(@Param("name") String name, Pageable p);

Whenever we add a custom API a /search endpoint gets added to the generated links. So if we call http://localhost:8080/subjects/search we will see a pagination capable endpoint:

"findByNameContaining" : {
  "href" : "http://localhost:8080/subjects/search/nameContains{?name,page,size,sort}",
  "templated" : true
}

All APIs that implement PagingAndSortingRepository will return a Page. If we need to return the list of the results from the Page, the getContent() API of Page provides the list of records fetched as a result of the Spring Data REST API.

The code in this section is available in the spring-data-rest project.

10. Convert a List into a Page

Let’s suppose that we have a Pageable object as input, but the information that we need to retrieve is contained in a list instead of a PagingAndSortingRepository. In these cases, we may need to convert a List into a Page.

For example, imagine that we have a list of results from a SOAP service:

List<Foo> list = getListOfFooFromSoapService();

We need to access the list in the specific positions specified by the Pageable object sent to us. So, let’s define the start index:

int start = (int) pageable.getOffset();

And the end index:

int end = (int) ((start + pageable.getPageSize()) > fooList.size() ? fooList.size()
  : (start + pageable.getPageSize()));

Having these two in place, we can create a Page to obtain the list of elements between them:

Page<Foo> page 
  = new PageImpl<Foo>(fooList.subList(start, end), pageable, fooList.size());

That’s it! We can return now page as a valid result.

And note that if we also want to give support for sorting, we need to sort the list before sub-listing it.

11. Conclusion

This article illustrated how to implement Pagination in a REST API using Spring, and discussed how to set up and test Discoverability.

If you want to go in-depth on pagination in the persistence level, check out my JPA or Hibernate pagination tutorials.

The implementation of all these examples and code snippets can be found in the GitHub project – this is a Maven-based project, so it should be easy to import and run as it is.