1. Michael Bayer
  2. sqlalchemy

Issues

Issue #3079 closed

select statement returning 0 rows generates: sqlalchemy.exc.ResourceClosedError: This result object does not return rows. It has been closed automatically.

mike_solomon
created an issue

Below is code that generates what is, IMO, a bug. To see the bug, comment out the line towards the end with the comment saying COMMENT ME OUT. That line will result in the issuing of a select request that, given how I understand SQL and sqlalchemy, should generate 0 rows. Instead, it causes a ResourceClosedError. I can use this select statement, however, to be inserted into other tables (see http://stackoverflow.com/questions/24094857/subqueries-for-filters-in-joined-sqlalchemy-statements/24159702#24159702). I think that the statement should return 0 rows instead of raising the error.

from sqlalchemy import Table, Column, String, Integer, MetaData, \
    select, func, ForeignKey, text
import sys
from functools import reduce

from sqlalchemy import create_engine
engine = create_engine('sqlite:///:memory:', echo=False)

metadata = MetaData()

linked_list = Table('linked_list', metadata,
    Column('id', Integer, primary_key = True),
    Column('at', Integer, nullable=False),
    Column('val', Integer, nullable=False),
    Column('next', Integer, ForeignKey('linked_list.at'))
)

refs = Table('refs', metadata,
    Column('id', Integer, primary_key = True),
    Column('ref', Integer, ForeignKey('linked_list.at')),
)

metadata.create_all(engine)
conn = engine.connect()

refs_al = refs.alias()

linked_list_m = select([
                    linked_list.c.at,
                    linked_list.c.val,
                    linked_list.c.next]).\
                    where(linked_list.c.at==refs_al.c.ref).\
                    cte(recursive=True)

llm_alias = linked_list_m.alias()
ll_alias = linked_list.alias()

linked_list_m = linked_list_m.union_all(
    select([
        llm_alias.c.at,
        ll_alias.c.val * llm_alias.c.val,
        ll_alias.c.next
    ]).
        where(ll_alias.c.at==llm_alias.c.next)
)


llm_alias_2 = linked_list_m.alias()

sub_statement = select([
            llm_alias_2.c.at,
            llm_alias_2.c.val]).\
        order_by(llm_alias_2.c.val.desc()).\
        limit(1)

def gen_statement(v) :
  return select([refs_al.c.ref, func.max(llm_alias_2.c.val)]).\
    select_from(
     refs_al.\
       join(llm_alias_2, onclause=refs_al.c.ref == llm_alias_2.c.at)).\
     group_by(refs_al.c.ref).where(llm_alias_2.c.val > v)

LISTS = [[2,4,4,11],[3,4,5,6]]

idx = 0
for LIST in LISTS :
  start = idx
  for x in range(len(LIST)) :
    ELT = LIST[x]
    conn.execute(linked_list.insert().\
      values(at=idx, val = ELT, next=idx+1 if x != len(LIST) - 1 else None))
    idx += 1
  conn.execute(refs.insert().values(ref=start))

print "LISTS:"
for LIST in LISTS :
  print "  ", LIST

def PRODUCT(L) : return reduce(lambda x,y : x*y, L, 1)
print "PRODUCTS OF LISTS:"
for LIST in LISTS :
  print "  ", PRODUCT(LIST)

for x in (345,355,365) :
  if x == 365 : continue # COMMENT ME OUT TO GET sqlalchemy.exc.ResourceClosedError
  statement_ = gen_statement(x)
  print "########"
  print "Lists that are greater than:", x
  conn.execute(statement_)
  allresults = conn.execute(statement_).fetchall()
  if len(allresults) == 0 :
    print "  /no results found/"
  else :
    for res in allresults :
      print res

print "########"

Comments (6)

  1. Michael Bayer repo owner

    this is a pysqlite bug. ALL DBAPI cursors must return a .description attribute when a SELECT has proceeded (please note the link, this is regardless of whether or not any rows are present). SQLAlchemy uses this as a guideline to know when the statement is one that returns rows, even one that returns zero rows (a zero-length result set still has column names).

    Below, a script illustrates this for some sample statements, and runs against psycopg2/postgresql as well as sqlite. Note that SQLAlchemy is not used, only plain DBAPIs psycopg2 and sqlite3. Only the last statement fails, and only on sqlite:

    s1 = "SELECT * FROM linked_list"
    s2 = "SELECT * FROM refs"
    s3 = """
    WITH RECURSIVE anon_2(at, val, next) AS
    (SELECT linked_list.at AS at, linked_list.val AS val, linked_list.next AS next
    FROM linked_list, refs AS refs_1
    WHERE linked_list.at = refs_1.ref
    UNION ALL SELECT anon_3.at AS at, linked_list_1.val * anon_3.val AS anon_4, linked_list_1.next AS next
    FROM anon_2 AS anon_3, linked_list AS linked_list_1
    WHERE linked_list_1.at = anon_3.next)
     SELECT refs_1.ref, max(anon_1.val) AS max_1
    FROM refs AS refs_1 JOIN anon_2 AS anon_1 ON refs_1.ref = anon_1.at
     GROUP BY refs_1.ref
    """
    
    import sqlite3
    sqlite_conn = sqlite3.connect(":memory:")
    
    import psycopg2
    pg_conn = psycopg2.connect(user='scott', password='tiger', database='test', host='localhost')
    
    for conn in (pg_conn, sqlite_conn):
        print ("Conn: ", conn)
        cursor = conn.cursor()
    
        cursor.execute("""
        CREATE TABLE linked_list (
            id INTEGER NOT NULL,
            at INTEGER NOT NULL,
            val INTEGER NOT NULL,
            next INTEGER,
            PRIMARY KEY (id),
            UNIQUE (at),
            FOREIGN KEY(next) REFERENCES linked_list (at)
        )
        """)
    
        cursor.execute("""
        CREATE TABLE refs (
            id INTEGER NOT NULL,
            ref INTEGER,
            PRIMARY KEY (id),
            FOREIGN KEY(ref) REFERENCES linked_list (at)
        )
    
        """)
    
        for statement in [s1, s2, s3]:
            print ("Statement: ", statement[0:30])
            cursor.execute(statement)
            assert cursor.description
    

    output:

    ('Conn: ', <connection object at 0x100559d90; dsn: 'dbname=test user=scott password=xxxxx host=localhost', closed: 0>)
    ('Statement: ', 'SELECT * FROM linked_list')
    ('Statement: ', 'SELECT * FROM refs')
    ('Statement: ', '\nWITH RECURSIVE anon_2(at, val')
    ('Conn: ', <sqlite3.Connection object at 0x100474118>)
    ('Statement: ', 'SELECT * FROM linked_list')
    ('Statement: ', 'SELECT * FROM refs')
    ('Statement: ', '\nWITH RECURSIVE anon_2(at, val')
    Traceback (most recent call last):
      File "test2.py", line 52, in <module>
        assert cursor.description
    AssertionError
    
  2. Michael Bayer repo owner

    this bug is very simple. pysqlite is clearly parsing the statement for a SELECT in order to detect a zero-row statement:

    import sqlite3
    conn = sqlite3.connect(":memory:")
    
    cursor = conn.cursor()
    cursor.execute("""
        create table foo (id integer primary key, data varchar(20))
    """)
    
    cursor.execute("""
        insert into foo (id, data) values (10, 'ten')
    """)
    
    cursor.execute("""
        with bar as (select * from foo)
        select * from bar where id = 10
    """)
    
    assert cursor.description
    
    
    cursor.execute("""
        with bar as (select * from foo)
        select * from bar where id = 11
    """)
    
    assert cursor.description
    
  3. Michael Bayer repo owner

    that said, if someone proposes a workaround in that issue such as adding a special comment to the statement, we can do that. reopen if such a workaround becomes possible.

  4. Log in to comment