Showing a list of data with Data Providers
- Lazy Loading Data to a Listing

Showing a list of data with Data Providers

A common pattern in applications is that the user is first presented with a list of items, from which she selects one or several items to continue working with. These items could be inventory records to survey, messages to respond to or blog drafts to edit or publish.

A listing is a component that displays one or several properties from a list of items, allowing the user to inspect the data, mark items as selected and in some cases even edit the item directly through the component. While each listing component has it’s own API for configuring exactly how the data is represented and how it can be manipulated, they all share the same mechanisms for receiving data to show. Namely, all officially supported listing components use a DataProvider to fetch items needed for display.

The items are generally either loaded directly from memory or lazy loaded from some kind of backend. Regardless of how the items are loaded, the component is configured with one or several callbacks that define how the item should be displayed.

In the following example, a ComboBox that lists status items is configured to use the Status.getLabel() method to represent each status. There is also a Grid, which is configured with one column from the person’s name and another showing the year of birth.

Java

ComboBox<Status> comboBox = new ComboBox<>();
comboBox.setItemLabelGenerator(Status::getLabel);
Grid<Person> grid = new Grid<>();
grid.addColumn(Person::getName).setHeader("Name");
grid.addColumn(person -> Integer.toString(person.getYearOfBirth()))
        .setHeader("Year of birth");

Note	In this example, it would not even be necessary to define any item caption provider for the combo box if `Status.toString()` would be implemented to return a suitable text. `ComboBox` is by default configured to use `toString()` for finding a caption to show.

After we have told the component how the data should be shown, we only need to give it some data to actually show. The easiest way of doing that is to directly pass the values to show to setItems.

Java

// Sets items as a collection
comboBox.setItems(EnumSet.allOf(Status.class));
// Sets items using varargs
grid.setItems(
  new Person("George Washington", 1732),
  new Person("John Adams", 1735),
  new Person("Thomas Jefferson", 1743),
  new Person("James Madison", 1751)
);

Listing components that allow the user to control the display order of the items are automatically able to sort data by any property as long as the property type implements Comparable.

We can also define a custom Comparator if we want to customize the way a specific column is sorted. The comparator can either be based on the item instances or on the values of the property that is being shown.

Java

grid.addColumn(Person::getName).setHeader("Name")
        // Override default natural sorting
        .setComparator(Comparator
                .comparing(person -> person.getName().toLowerCase()));

Note	This kind of sorting is only supported for in-memory data. Sorting with data that is lazy loaded from a backend is described later in this chapter.

Instead of directly assigning the item collection as the items that a component should be using, we can instead create a ListDataProvider that contains the items. One list data provider instance can be shared between different components to make them show the same data. The instance can be further configured to filter out some of the items or to present them in a specific order.

For components like Grid that can be separately configured to sort data in a specific way, the sorting configured in the data provider is only used as a fallback. The fallback is used if no sorting is defined through the component and to define the order between items that are considered to be the same according to the component’s sorting. All components will automatically update themselves when the sorting of the data provider is changed.

Java

ListDataProvider<Person> dataProvider =
  DataProvider.ofCollection(persons);
dataProvider.setSortOrder(Person::getName,
  SortDirection.ASCENDING);
Grid<Person> grid = new Grid<>();
// The grid shows the persons sorted by name
grid.setDataProvider(dataProvider);
// Makes the combo box show persons in descending order
button.addClickListener(event -> {
  dataProvider.setSortOrder(Person::getName,
    SortDirection.DESCENDING);
});

You can configure the data provider to always apply some specific filter to limit which items are shown or to make it filter by data that is not included in the displayed item caption.

Java

ListDataProvider<Person> dataProvider =
  DataProvider.ofCollection(persons);
ComboBox<Person> comboBox = new ComboBox<>();
comboBox.setDataProvider(dataProvider);
departmentSelect.addValueChangeListener(event -> {
  Department selectedDepartment = event.getValue();
  if (selectedDepartment != null) {
    dataProvider.setFilterByValue(
      Person::getDepartment,
      selectedDepartment);
  } else {
    dataProvider.clearFilters();
  }
});

In this example, the department selected in the departmentSelect component is used to dynamically change which persons are shown in the combo box. In addition to setFilterByValue, it is also possible to set a filter based on a predicate that tests each item or the value of some specific property in the item. Multiple filters can also be stacked by using addFilter methods instead of setFilter.

The listing component cannot automatically know about changes to the list of items or to any individual item. We must notify the data provider when items are changed, added or removed so that components using the data will show the new values.

Java

ListDataProvider<Person> dataProvider =
  new ListDataProvider<>(persons);
Button addPersonButton = new Button("Add person",
  clickEvent -> {
    persons.add(new Person("James Monroe", 1758));
    dataProvider.refreshAll();
});
Button modifyPersonButton = new Button("Modify person",
  clickEvent -> {
    Person personToChange = persons.get(0);
    personToChange.setName("Changed person");
    dataProvider.refreshItem(personToChange);
});

Lazy Loading Data to a Listing

All the previous examples have shown cases with a limited amount of data that can be loaded as item instances in memory. There are also situations where it is more efficient to only load the items that will currently be displayed. This includes situations where all available data would use lots of memory or when it would take a long time to load all the items.

Note	Regardless of how we make the items available to the listing component on the server, components like `Grid` will always take care of only sending the currently needed items to the browser.

For example, if we have the following existing backend service that fetches items from a database or a REST service .

Java

public interface PersonService {
  List<Person> fetchPersons(int offset, int limit);
  int getPersonCount();
}

To use this service with a listing component, we need to define one callback for loading specific items and one callback for finding how many items are currently available. Information about which items to fetch as well as some additional details are made available in a Query object that is passed to both callbacks.

Java

DataProvider<Person, Void> dataProvider = DataProvider.fromCallbacks(
  // First callback fetches items based on a query
  query -> {
    // The index of the first item to load
    int offset = query.getOffset();
    // The number of items to load
    int limit = query.getLimit();
    List<Person> persons = getPersonService()
            .fetchPersons(offset, limit);
    return persons.stream();
  },
  // Second callback fetches the number of items for a query
  query -> getPersonService().getPersonCount());
);
Grid<Person> grid = new Grid<>();
grid.setDataProvider(dataProvider);
// Columns are configured in the same way as before

Note

The results of the first and second callback must be symmetric so that fetching all available items using the first callback returns the number of items indicated by the second callback. Thus if you impose any restrictions on e.g. a database query in the first callback, you must also add the same restrictions for the second callback.

Note	The second type parameter of `DataProvider` defines how the provider can be filtered. In this case the filter type is `Void`, meaning that it doesn’t support filtering. Backend filtering will be covered later in this chapter.

Sorting

It is not practical to order items based on a Comparator when the items are loaded on demand, since it would require all items to be loaded and inspected.

Each backend has its own way of defining how the fetched items should be ordered, but they are in general based on a list of property names and information on whether ordering should be ascending or descending.

As an example, there could be a service interface which looks like the following.

Java

public interface PersonService {
  List<Person> fetchPersons(
    int offset,
    int limit,
    List<PersonSort> sortOrders);
  int getPersonCount();
  PersonSort createSort(
    String propertyName,
    boolean descending);
}

With the above service interface, our data source can be enhanced to convert the provided sorting options into a format expected by the service. The sorting options set through the component will be available through Query.getSortOrders().

Java

DataProvider<Person, Void> dataProvider = DataProvider.fromCallbacks(
  query -> {
    List<PersonSort> sortOrders = new ArrayList<>();
    for(SortOrder<String> queryOrder : query.getSortOrders()) {
      PersonSort sort = getPersonService().createSort(
        // The name of the sorted property
        queryOrder.getSorted(),
        // The sort direction for this property
        queryOrder.getDirection() == SortDirection.DESCENDING);
      sortOrders.add(sort);
    }
    return getPersonService().fetchPersons(
        query.getOffset(),
        query.getLimit(),
        sortOrders
      );
  },
  // The number of persons is the same regardless of ordering
  query -> getPersonService().getPersonCount()
);

We also need to configure our grid so that it can know what property name should be included in the query when the user wants to sort by a specific column. When a data source that does lazy loading is used, Grid and other similar components will only let the user sort by columns for which a sort property name is provided.

Java

Grid<Person> grid = new Grid<>();
grid.setDataProvider(dataProvider);
// Will be sortable by the user
// When sorting by this column, the query will have a SortOrder
// where getSorted() returns "name"
grid.addColumn(Person::getName).setHeader("Name")
                .setSortProperty("name");
// Will not be sortable since no sorting info is given
grid.addColumn(Person::getYearOfBirth).setHeader("Year of birth");

There might also be cases where a single property name is not enough for sorting. This might be the case if the backend needs to sort by multiple properties for one column in the user interface or if the backend sort order should be inverted compared to the sort order defined by the user. In such cases, we can define a callback that generates suitable SortOrder values for the given column.

Java

grid.addColumn(person -> person.getName() + " " + person.getLastName())
        .setHeader("Name").setSortOrderProvider(
                // Sort according to last name, then first name
                direction -> Stream.of(
                        new QuerySortOrder("lastName", direction),
                        new QuerySortOrder("firstName", direction)));

Filtering

Different types of backends support filtering in different ways. Some backends support no filtering at all, some support filtering by a single value of some specific type and some have a complex structure of supported filtering options.

A DataProvider<Person, String> accepts one string to filter by through the query. It’s up to the data provider implementation to decide what it does with that filter value. It might, for instance, look for all persons with a name beginning with the provided string.

A listing component that lets the user control how the displayed data is filtered has some specific filter type that it uses. For ComboBox, the filter is the String that the user has typed into the search field. This means that ComboBox can only be used with a data provider whose filtering type is String.

To use a data provider that filters by some other type, you need to use the withConvertedFilter. This method creates a new data provider that uses the same data but a different filter type; converting the filter value before passing it to the original data provider instance.

We might, for instance, have a data provider that finds any person where the name contains any of the strings in a set. To use that data provider with a combo box, we need to define a converter that receives a single string from the combo box and creates a set of string that the data provider expects.

Java

DataProvider<Person, Set<String>> personProvider = getProvider();
ComboBox<Person> comboBox = new ComboBox<>();
DataProvider<Person, String> converted = personProvider
        .withConvertedFilter(
                filterText -> Collections.singleton(filterText));
comboBox.setDataProvider(converted);

The filter value passed through the query does typically originate from a component such as ComboBox that lets the user filter by some value. It is also possible to create a data provider wrapper that allows programmatically setting the filter value to include in the query.

You can use the withConfigurableFilter method on a data provider to create a data provider wrapper that allows configuring the filter that is passed through the query. All components that use a data provider will refresh their data when a new filter is set.

Java

DataProvider<Person, String> personProvider = getPersonProvider();
ConfigurableFilterDataProvider<Person, Void, String> wrapper =
  personProvider.withConfigurableFilter();
Grid<Person> grid = new Grid<>();
grid.setDataProvider(personProvider);
grid.addColumn(Person::getName).setHeader("Name");
searchField.addValueChangeListener(event -> {
  String filter = event.getValue();
  if (filter.trim().isEmpty()) {
    // null disables filtering
    filter = null;
  }
  wrapper.setFilter(filter);
});

Note that the filter type of the wrapper instance is Void, which means that the data provider doesn’t support any further filtering through the query. It’s therefore not possible to use the data provider with a combo box.

There is an overload of withConfigurableFilter that uses a callback for combining the configured filter value with a filter value from the query. We can thus wrap our data provider that filters by a set of strings to create a data provider that combines a string from a combo box with a set of strings that are separately configured.

Java

DataProvider<Person, Set<String>> personProvider = getPersonsProvider();
ConfigurableFilterDataProvider<Person, String, Set<String>> wrapper =
  personProvider.withConfigurableFilter(
    (String queryFilter, Set<String> configuredFilters) -> {
      Set<String> combinedFilters = new HashSet<>();
      combinedFilters.addAll(configuredFilters);
      combinedFilters.add(queryFilter);
      return combinedFilters;
    }
  );
wrapper.setFilter(Collections.singleton("John"));
ComboBox<Person> comboBox = new ComboBox<>();
comboBox.setDataProvider(wrapper);

In this case, wrapper supports a single string as the query filter and Set<String> trough setFilter. The callback combines both into one Set<String> that will be in the query passed to personProvider.

To create a data provider that supports filtering, you only need to look for a filter in the provided query and use that filter when fetching and counting items. withConfigurableFilter and withConvertedFilter are automatically implemented for you.

As an example, our service interface with support for filtering could look like this. Ordering support has been omitted in this example to keep focus on filtering.

Java

public interface PersonService {
  List<Person> fetchPersons(
    int offset,
    int limit,
    String namePrefix);
  int getPersonCount(String namePrefix);
}

A data provider using this service could use String as its filtering type. It would then look for a string to filter by in the query and pass it to the service method.

Java

DataProvider<Person, String> dataProvider = DataProvider
        .fromFilteringCallbacks(query -> {
            // getFilter returns Optional<String>
            String filter = query.getFilter().orElse(null);
            return getPersonService().fetchPersons(query.getOffset(),
                    query.getLimit(), filter).stream();
        }, query -> {
            String filter = query.getFilter().orElse(null);
            return getPersonService().getPersonCount(filter);
        });

If we instead have a service that expects multiple different filtering parameters, we can use two different alternatives depending on how the data provider would be used. Both cases would be based on this example service API:

Java

public interface PersonService {
  List<Person> fetchPersons(
    int offset,
    int limit,
    String namePrefix,
    Department department);
  int getPersonCount(
    String namePrefix,
    Department department);
}

The first approach would be to define a simple wrapper class that combines both filter parameters into one instance.

Java

public class PersonFilter {
  public final String namePrefix;
  public final Department department;
  public PersonFilter(String namePrefix, Department department) {
    this.namePrefix = namePrefix;
    this.department = department;
  }
}

We can then define a data provider that is natively filtered by PersonFilter.

Java

DataProvider<Person, PersonFilter> dataProvider = DataProvider
        .fromFilteringCallbacks(query -> {
            PersonFilter filter = query.getFilter().orElse(null);
            return getPersonService()
                    .fetchPersons(query.getOffset(), query.getLimit(),
                            filter != null ? filter.namePrefix : null,
                            filter != null ? filter.department : null)
                    .stream();
        }, query -> {
            PersonFilter filter = query.getFilter().orElse(null);
            return getPersonService().getPersonCount(
                    filter != null ? filter.namePrefix : null,
                    filter != null ? filter.department : null);
        });

This data provider can then be used in different ways with withConvertedFilter or withConfigurableFilter.

Java

// For use with ComboBox without any department filter
DataProvider<Person, String> onlyString = dataProvider.withConvertedFilter(
  filterString -> new PersonFilter(filterString, null)
);
// For use with some external filter, e.g. a search form
ConfigurableFilterDataProvider<Person, Void, PersonFilter> everythingConfigurable =
  dataProvider.withConfigurableFilter();
everythingConfigurable.setFilter(
  new PersonFilter(someText, someDepartment));
// For use with ComboBox and separate department filtering
ConfigurableFilterDataProvider<Person, String, Department> mixed =
  dataProvider.withConfigurableFilter(
    // Can be shortened as PersonFilter::new
    (filterText, department) -> {
      return new PersonFilter(filterText, department);
    }
  );
mixed.setFilter(someDepartment);

The other alternative for using this kind of service API is to define your own data provider subclass that has setter methods for the filter parameters that should not be passed as the query filter. We might for instance want to receive the name filter through the query from a combo box while the department to filter by is set from application code. We must remember to call refreshAll() when the department filter has been changed so that any components can know that they should fetch new data to show.

Java

public class PersonDataProvider
  extends AbstractBackEndDataProvider<Person, String> {
  private Department departmentFilter;
  public void setDepartmentFilter(Department department) {
    this.departmentFilter = department;
    refreshAll();
  }
  @Override
  protected Stream<Person> fetchFromBackEnd(Query<Person, String> query) {
    return getPersonService().fetchPersons(
      query.getOffset(),
      query.getLimit(),
      query.getFilter().orElse(null),
      departmentFilter
    ).stream();
  }
  @Override
  protected int sizeInBackEnd(Query<Person, String> query) {
    return getPersonService().getPersonCount(
      query.getFilter().orElse(null),
      departmentFilter
    );
  }
}

Refreshing

When your application makes changes to the data that is in your backend, you might need to make sure all parts of the application are aware of these changes. All data providers have the refreshAll and refreshItem methods. These methods can be used when data in the backend has been updated.

For example Spring Data gives you new instances with every request, and making changes to the repository will make old instances of the same object “stale”. In these cases you should inform any interested component by calling dataProvider.refreshItem(newInstance). This can work out of the box, if your beans have equals and hashCode implementations that check if the objects represent the same data. Since that is not always the case, the user of a CallbackDataProvider can give it a ValueProvider that will provide a stable ID for the data objects. This is usually a method reference, eg. Person::getId.

As an example, our service interface has an update method that returns a new instance of the item. Other functionality has been omitted to keep focus on the updating.

Java

public interface PersonService {
  Person save(Person person);
}

Part of the application code wants to update a persons name and save it to the backend.

Java

DataProvider<Person, String> allPersonsWithId = new CallbackDataProvider<>(
  fetchCallback, sizeCallback, Person::getId);
Grid<Person> persons = new Grid<>();
persons.setDataProvider(allPersonsWithId);
persons.addColumn(Person::getName).setHeader("Name");
Button modifyPersonButton = new Button("Modify person", clickEvent -> {
    Person personToChange = allPersonsWithId.fetch(
            new Query<>(0, 1, Collections.emptyList(), null, null))
            .findFirst().get();
    personToChange.setName("Changed person");
    Person newInstance = service.save(personToChange);
    dataProvider.refreshItem(newInstance);
});