ActiveRecord::ConnectionAdapters::ConnectionPool (original) (raw)

Active Record Connection Pool

Connection pool base class for managing Active Record database connections.

Introduction

A connection pool synchronizes thread access to a limited number of database connections. The basic idea is that each thread checks out a database connection from the pool, uses that connection, and checks the connection back in. ConnectionPool is completely thread-safe, and will ensure that a connection cannot be used by two threads at the same time, as long as ConnectionPool’s contract is correctly followed. It will also handle cases in which there are more threads than connections: if all connections have been checked out, and a thread tries to checkout a connection anyway, then ConnectionPool will wait until some other thread has checked in a connection, or the checkout_timeout has expired.

Obtaining (checking out) a connection

Connections can be obtained and used from a connection pool in several ways:

  1. Simply use ActiveRecord::Base.lease_connection. When you’re done with the connection(s) and wish it to be returned to the pool, you call ActiveRecord::Base.connection_handler.clear_active_connections!. This is the default behavior for Active Record when used in conjunction with Action Pack’s request handling cycle.
  2. Manually check out a connection from the pool with ActiveRecord::Base.connection_pool.checkout. You are responsible for returning this connection to the pool when finished by calling ActiveRecord::Base.connection_pool.checkin(connection).
  3. Use ActiveRecord::Base.connection_pool.with_connection(&block), which obtains a connection, yields it as the sole argument to the block, and returns it to the pool after the block completes.

Connections in the pool are actually AbstractAdapter objects (or objects compatible with AbstractAdapter’s interface).

While a thread has a connection checked out from the pool using one of the above three methods, that connection will automatically be the one used by ActiveRecord queries executing on that thread. It is not required to explicitly pass the checked out connection to Rails models or queries, for example.

Options

There are several connection-pooling-related options that you can add to your database connection configuration:

Namespace

Methods

A

C

D

F

I

L

N

R

S

W

Included Modules

Constants

WeakThreadKeyMap = ObjectSpace::WeakKeyMap

Attributes

[R] async_executor
[RW] automatic_reconnect
[RW] checkout_timeout
[R] db_config
[R] pool_config
[R] reaper
[R] role
[R] shard
[R] size

Class Public methods

install_executor_hooks(executor = ActiveSupport::Executor)Link

Source: show | on GitHub

def install_executor_hooks(executor = ActiveSupport::Executor) executor.register_hook(ExecutorHooks) end

Creates a new ConnectionPool object. pool_config is a PoolConfig object which describes database connection information (e.g. adapter, host name, username, password, etc), as well as the maximum size for this ConnectionPool.

The default ConnectionPool maximum size is 5.

Source: show | on GitHub

def initialize(pool_config) super()

@pool_config = pool_config @db_config = pool_config.db_config @role = pool_config.role @shard = pool_config.shard

@checkout_timeout = db_config.checkout_timeout @idle_timeout = db_config.idle_timeout @size = db_config.pool

@leases = LeaseRegistry.new

@connections = [] @automatic_reconnect = true

@now_connecting = 0

@threads_blocking_new_connections = 0

@available = ConnectionLeasingQueue.new self @pinned_connection = nil @pinned_connections_depth = 0

@async_executor = build_async_executor

@schema_cache = nil

@reaper = Reaper.new(self, db_config.reaping_frequency) @reaper.run end

Instance Public methods

Source: show | on GitHub

def active_connection? connection_lease.connection end

Check-in a database connection back into the pool, indicating that you no longer need this connection.

conn: an AbstractAdapter object, which was obtained by earlier by calling checkout on this pool.

Source: show | on GitHub

def checkin(conn) return if @pinned_connection.equal?(conn)

conn.lock.synchronize do synchronize do connection_lease.clear(conn)

  conn._run_checkin_callbacks do
    conn.expire
  end

  @available.add conn
end

end end

checkout(checkout_timeout = @checkout_timeout)Link

Check-out a database connection from the pool, indicating that you want to use it. You should call checkin when you no longer need this.

This is done by either returning and leasing existing connection, or by creating a new connection and leasing it.

If all connections are leased and the pool is at capacity (meaning the number of currently leased connections is greater than or equal to the size limit set), an ActiveRecord::ConnectionTimeoutError exception will be raised.

Returns: an AbstractAdapter object.

Raises:

Source: show | on GitHub

def checkout(checkout_timeout = @checkout_timeout) return checkout_and_verify(acquire_connection(checkout_timeout)) unless @pinned_connection

@pinned_connection.lock.synchronize do synchronize do

  if @pinned_connection
    @pinned_connection.verify!
    
    
    unless @connections.include?(@pinned_connection)
      @connections << @pinned_connection
    end

    @pinned_connection
  else
    checkout_and_verify(acquire_connection(checkout_timeout))
  end
end

end end

clear_reloadable_connections(raise_on_acquisition_timeout = true)Link

Clears the cache which maps classes and re-connects connections that require reloading.

Raises:

Source: show | on GitHub

def clear_reloadable_connections(raise_on_acquisition_timeout = true) with_exclusively_acquired_all_connections(raise_on_acquisition_timeout) do synchronize do @connections.each do |conn| if conn.in_use? conn.steal! checkin conn end conn.disconnect! if conn.requires_reloading? end @connections.delete_if(&:requires_reloading?) @available.clear end end end

Clears the cache which maps classes and re-connects connections that require reloading.

The pool first tries to gain ownership of all connections. If unable to do so within a timeout interval (default duration is spec.db_config.checkout_timeout * 2 seconds), then the pool forcefully clears the cache and reloads connections without any regard for other connection owning threads.

Source: show | on GitHub

def clear_reloadable_connections! clear_reloadable_connections(false) end

Returns true if a connection has already been opened.

Source: show | on GitHub

def connected? synchronize { @connections.any?(&:connected?) } end

Returns an array containing the connections currently in the pool. Access to the array does not require synchronization on the pool because the array is newly created and not retained by the pool.

However; this method bypasses the ConnectionPool’s thread-safe connection access pattern. A returned connection may be owned by another thread, unowned, or by happen-stance owned by the calling thread.

Calling methods on a connection without ownership is subject to the thread-safety guarantees of the underlying method. Many of the methods on connection adapter classes are inherently multi-thread unsafe.

Source: show | on GitHub

def connections synchronize { @connections.dup } end

disconnect(raise_on_acquisition_timeout = true)Link

Disconnects all connections in the pool, and clears the pool.

Raises:

Source: show | on GitHub

def disconnect(raise_on_acquisition_timeout = true) with_exclusively_acquired_all_connections(raise_on_acquisition_timeout) do synchronize do @connections.each do |conn| if conn.in_use? conn.steal! checkin conn end conn.disconnect! end @connections = [] @leases.clear @available.clear end end end

Disconnects all connections in the pool, and clears the pool.

The pool first tries to gain ownership of all connections. If unable to do so within a timeout interval (default duration is spec.db_config.checkout_timeout * 2 seconds), then the pool is forcefully disconnected without any regard for other connection owning threads.

flush(minimum_idle = @idle_timeout)Link

Disconnect all connections that have been idle for at least minimum_idle seconds. Connections currently checked out, or that were checked in less than minimum_idle seconds ago, are unaffected.

Source: show | on GitHub

def flush(minimum_idle = @idle_timeout) return if minimum_idle.nil?

idle_connections = synchronize do return if self.discarded? @connections.select do |conn| !conn.in_use? && conn.seconds_idle >= minimum_idle end.each do |conn| conn.lease

  @available.delete conn
  @connections.delete conn
end

end

idle_connections.each do |conn| conn.disconnect! end end

Disconnect all currently idle connections. Connections currently checked out are unaffected.

Retrieve the connection associated with the current thread, or call checkout to obtain one if necessary.

lease_connection can be called any number of times; the connection is held in a cache keyed by a thread.

Source: show | on GitHub

def lease_connection lease = connection_lease lease.sticky = true lease.connection ||= checkout end

Recover lost connections for the pool. A lost connection can occur if a programmer forgets to checkin a connection at the end of a thread or a thread dies unexpectedly.

Source: show | on GitHub

def reap stale_connections = synchronize do return if self.discarded? @connections.select do |conn| conn.in_use? && !conn.owner.alive? end.each do |conn| conn.steal! end end

stale_connections.each do |conn| if conn.active? conn.reset! checkin conn else remove conn end end end

release_connection(existing_lease = nil)Link

Signal that the thread is finished with the current connection. release_connection releases the connection-thread association and returns the connection to the pool.

This method only works for connections that have been obtained through lease_connection or with_connection methods, connections obtained through checkout will not be automatically released.

Source: show | on GitHub

def release_connection(existing_lease = nil) if conn = connection_lease.release checkin conn return true end false end

Remove a connection from the connection pool. The connection will remain open and active but will no longer be managed by this pool.

Source: show | on GitHub

def remove(conn) needs_new_connection = false

synchronize do remove_connection_from_thread_cache conn

@connections.delete conn
@available.delete conn


needs_new_connection = @available.any_waiting?

end

bulk_make_new_connections(1) if needs_new_connection end

Source: show | on GitHub

def schema_cache @schema_cache ||= BoundSchemaReflection.new(schema_reflection, self) end

Source: show | on GitHub

def schema_reflection=(schema_reflection) pool_config.schema_reflection = schema_reflection @schema_cache = nil end

Returns the connection pool’s usage statistic.

ActiveRecord::Base.connection_pool.stat # => { size: 15, connections: 1, busy: 1, dead: 0, idle: 0, waiting: 0, checkout_timeout: 5 }

Source: show | on GitHub

def stat synchronize do { size: size, connections: @connections.size, busy: @connections.count { |c| c.in_use? && c.owner.alive? }, dead: @connections.count { |c| c.in_use? && !c.owner.alive? }, idle: @connections.count { |c| !c.in_use? }, waiting: num_waiting_in_queue, checkout_timeout: checkout_timeout } end end

Yields a connection from the connection pool to the block. If no connection is already checked out by the current thread, a connection will be checked out from the pool, yielded to the block, and then returned to the pool when the block is finished. If a connection has already been checked out on the current thread, such as via lease_connection or with_connection, that existing connection will be the one yielded and it will not be returned to the pool automatically at the end of the block; it is expected that such an existing connection will be properly returned to the pool by the code that checked it out.

Source: show | on GitHub

def with_connection(prevent_permanent_checkout: false) lease = connection_lease sticky_was = lease.sticky lease.sticky = false if prevent_permanent_checkout

if lease.connection begin yield lease.connection ensure lease.sticky = sticky_was if prevent_permanent_checkout && !sticky_was end else begin yield lease.connection = checkout ensure lease.sticky = sticky_was if prevent_permanent_checkout && !sticky_was release_connection(lease) unless lease.sticky end end end