Infinite scrolling pagination with Rails, Hotwire, and Turbo

One of the reasons we love Ruby on Rails is that even though there is often the “rails way” to solve a problem, you can still take advantage of other great approaches.

Infinite scrolling is one such problem that has a variety of implementations, but none that quite worked the way we wanted it to. Infinite scrolling is the design pattern you often see used for long lists of items. Rather than implement traditional pagination, new items are loaded at the end of the list—often automatically when the user reaches the end, or via a “load more” button.

Here’s how we would do it with two different Hotwire approaches:

1. Turbo Streams (autoload content while scrolling)
2. Turbo Frames (“load more” button)

We’ve written in the past on the use cases and differences of turbo streams and turbo frames. We won’t talk about the pros and cons of the infinite scroll vs. pagination patterns in this post, but you will learn to implement different approaches with Hotwire.

Initial setup

This tutorial assumes some basic rails knowledge. You’ll need to get an app up and running. Then, scaffold some comments and posts, and add some seeds:

# db/seeds.rb
100.times do
  Post.create(title: Faker::Music.band, image: Faker::Avatar.image)
  Comment.create(body: Faker::Quote.famous_last_words)
end

bundle add faker
bundle add pagy
rails g scaffold comment body:text --no-helper --no-assets --no-controller-specs --no-view-specs --no-test-framework --no-jbuilder
rails g scaffold post title image --no-helper --no-assets --no-controller-specs --no-view-specs --no-test-framework --no-jbuilder
rails db:migrate
rails db:seed

We’re using faker to generate some placeholder data, and Pagy for pagination.

Next, include “gem pagy” to the helper:

# app/helpers/application_helper.rb
  include Pagy::Frontend

And again to the controller:

# app/controllers/application_controller.rb
  include Pagy::Backend

Turbo Streams: Click to load more

For the first approach, we’ll implement a classic “load more” button. This lets the user to manually append the next page to the end of the list. Enable Pagy countless; it is a recommended extra for that makes infinite scrolling easier in Pagy.

### config/initializers/pagy.rb
require 'pagy/extras/countless'

In the comments view, add a “button_to” to the next page using Pagy with the method POST.

# app/views/comments/index.html.erb
  <div id="comments">
    <%= render @comments %>
  </div>

  <div id="next_link">
    <%= button_to "next", pagy_url_for(@pagy, @pagy.next) %>
  </div>

Next, add pagination to the controller and configure it to respond to POST requests for turbo_streams:

# app/controllers/comments_controller.rb
class CommentsController < ApplicationController
  def index
    @pagy, @comments = pagy_countless(Comment.order(created_at: :desc), items: 5)

    respond_to do |format|
      format.html # GET
      format.turbo_stream # POST
    end
  end
end

Allow index_path to respond to POST, not only GET requests:

# config/routes.rb
  resources :comments do
    collection do
      post :index
    end
  end

Finally, add turbo stream’s append and update methods. Append will add the next page’s comments to the current comments render, and update will adjust the “next/load more” button to use a new URL now that the current page includes two pages of results.

# app/views/comments/index.turbo_stream.erb
<%= turbo_stream.append "comments" do %>
  <%= render partial: "comment", collection: @comments %>
<% end %>

<%= turbo_stream.update "next_link" do %>
  <% if @pagy.next.present? %>
    <%= button_to "next", pagy_url_for(@pagy, @pagy.next) %>
  <% end %>
<% end %>

Here are the results:

You can watch a full, detailed walkthrough of this approach at Ruby on Rails #67 Streams: Infinite Scroll Pagination, and view the full source on GitHub at Rails 7 app Turbo Streams pagination.

Turbo Frames: Auto-loading Infinite scroll

Now that we can load in the new items manually, it’s time to make them automatically load in when the user reaches the end of the list. By default, a lazy_loading turbo_frame only loads when it becomes visible in the DOM, as I’ve shown in my example on loading frames in a dropdown.

Knowing this, we don’t need to write any Javascript to observe whether the bottom of the page has become visible on screen! The built in lazy load functionality handles it for us. All we have to do is load a turbo_frame with the next page when we scroll to the bottom of current page. Let’s do it.

First, add pagination to the controller and set it up to respond with only the scrollable area when the incoming request has a pagination param.

# app/controllers/posts_controller.rb
  def index
    @pagy, @posts = pagy_countless(Post.order(created_at: :desc), items: 5)

    render "scrollable_list" if params[:page]
  end

Next, create a turbo frame with a unique ID. We’re generating an ID based on the ID of the next page. Set loading to “lazy” and target to “_top”. You can also experiment with the autoscroll property, but it doesn’t change the UX much in this use case.

# app/views/posts/_next_page.html.erb
<%= turbo_frame_tag(
      "posts-page-#{@pagy.next}",
      # autoscroll: true,
      loading: :lazy,
      src: pagy_url_for(@pagy, @pagy.next),
      target: "_top"
    ) do %>
  Loading...
<% end %>

Now in the posts view, render the posts collection and the next_page view that you just set up.

# app/views/posts/index.html.erb
<%= render partial: "post", collection: @posts %>
<%= render partial: "next_page" %>

Finally, set up the scrollable list by setting the turbo frame tag to the ID for the current page. Then, render the list and render the new frame for the next page below it.

# app/views/posts/scrollable_list.html.erb
<%= turbo_frame_tag "posts-page-#{@pagy.page}" do %>
  <%= render partial: "post", collection: @posts %>
  <%= render partial: "next_page" %>
<% end %>

The result looks like this:

You can watch a full, detailed walkthrough of this approach at Ruby on Rails #68 Frames: Infinite Scroll Pagination, and view the code on GitHub at Rails 7 app Turbo Frames pagination.

As an aside, here are a few of the pagy helpers to make working with it easier:

= @pagy.page # current page (1)
= @pagy.next # next page (2)
= pagy_next_link(@pagy, text: 'More...') # link to next page
= pagy_url_for(@pagy, @pagy.next) # url to next page (/posts&page=2)

# config/initializers/pagy.rb

# to enable pagy_next_link
# https://ddnexus.github.io/pagy/extras/support.html#gsc.tab=0
# require 'pagy/extras/support'

 

Frames + Streams: Perfected Auto-loading infinite scroll

If we combine the techniques thus far, we end up with code that looks a bit like this:

### app/views/posts/index.html.erb
  <div id="posts"></div>
  <%= turbo_frame_tag "pagination", src: posts_path(format: :turbo_stream), loading: :lazy %>

### app/views/posts/index.turbo_stream.erb
<%= turbo_stream.append "posts" do %>
   <%= render partial: "posts/post", collection: @posts %>
 <% end %>
<% if @pagy.next.present? %>
  <%= turbo_stream.replace "pagination" do %>
    <%= turbo_frame_tag "pagination", src: posts_path(page: @pagy.next, format: :turbo_stream), loading: :lazy %>
  <% end %>
<% end %>

Bonus: Cursor-based pagination without a pagination gem

One thing that often trips people up is paginating lists where the number of items can change while paginating. For example, let’s assume you have the following set of posts:

[A, B, C, D, E, F]

You paginate at 3 per page, set up a ‘load more’ button interaction, and request the first page. This leaves the first page as:

[A, B, C]

But what happens if you delete B though a turbo stream request? Your page now shows:

[A, C]

This updates the database as well, and it now contains:

[A, C, D, E, F]

While the first page shows 2 results instead of 3, the second page is using the newly updated database. Now when you request the next page of results, it it thinks D is now part of page 1 and only appends the next three results (in this case, there are only two more):

[A, C, E, F]

See the problem? We have an initial batch of results based on the original database, plus our deletion, and a second set of results based on the updated database. This leaves our page missing the item in the middle.

With cursor based paging, you say “give me the item after ‘C'” rather than a particular page number. Given this set we would get the correct result of

[A, C, D, E, F]

Ruby Implementation

Any unique key that you can “order by” can work as a cursor. In this example I use the ID since it’s the most convenient. The next item will always have an ID greater than the cursor, so even if the cursor item is removed, the next item is returned correctly. As you can see from these requirements, sorting by properties like date require a bit more thought than simple pagination, so its best to only use this approach when needed.

# app/controllers/posts_controller.rb
  POSTS_PER_PAGE = 5

  def index
    @cursor = (params[:cursor] || "0").to_i
    @posts = Post.all.where("id > ?", @cursor).order(:id).take(POSTS_PER_PAGE)
    @next_cursor = @posts.last&.id
    @more_pages = @next_cursor.present? && @posts.count == POSTS_PER_PAGE

    render "scrollable_list" if params[:cursor]
  end

Theres an edge case in the above where the last page contains a full page, but all that happens in the final load is a page with no items. In this instance that isn’t a big deal, but it is something to be aware of.

# app/views/posts/index.html.erb
<%= render partial: "post", collection: @posts %>
<%= render partial: "next_page" %>

# app/views/posts/_next_page.html.erb
<% if @more_pages %>
  <%= turbo_frame_tag(
        "posts-page-#{@next_cursor}",
        autoscroll: true,
        loading: :lazy,
        src: posts_path(cursor: @next_cursor),
        target: "_top"
      ) do %>
    Loading...
  <% end %>
<% end %>

# app/views/posts/scrollable_list.html.erb
<%= turbo_frame_tag "posts-page-#{@cursor}" do %>
  <%= render partial: "post", collection: @posts %>
  <%= render partial: "next_page" %>
<% end %>

Turbocharged pagination

Hopefully this has helped you better understand some approaches to pagination, taking full advantage of Hotwire’s Turbo streams and frames. If you haven’t already, check out our other Rails, Hotwire, and Turbo articles on our blog.