[FIXED] Do async responses get stored or processed immediately?

Issue

Say I'm communicating with an API which, for any valid parameter, responds with a very long list. I want to extract the first element of this list for several parameters. My code looks like this (feedback welcome):

import asyncio
import aiohttp

URL = 'https://api2.binance.com/api/v3/trades?symbol='

symbols = ['BTCUSDT', 'ETHBTC', 'ETHUSDT']

async def get_trades(symbols):
    async with aiohttp.ClientSession() as session:
        tasks = [session.get(URL + symbol) for symbol in symbols]
        all_trades = await asyncio.gather(*tasks)
        trades = [(await x.json())[0] for x in all_trades]
        return trades

await get_trades(symbols)

My question is: When running get_trades, do the full lists for all parameters get stored somewhere simultaneously, or is each list dealt with and then discarded immediately?

In other words, is all_trades something like a list of lists which is first completely created and then iterated over (to extract the first elements) or is it more like an abstract object instructing the compiler what to do when creating trades?

I'm asking because, if the responses are indeed first all stored together, this would of course take up unnecessary space as I'm only interested in one element of each list. In this case, how could I modify my code to get rid of this inefficiency?

Solution

Yes, all of the responses will be stored in all_trades, before being processed in the next lines. To avoid that, you can discard what you don't need before it is collected from all tasks. Approximately this way:

async def get_one_trade(session, symbol):
    x = await session.get(URL + symbol)
    return (await x.json())[0]  # discard the rest


async def get_trades(symbols):
    async with aiohttp.ClientSession() as session:
        tasks = [get_one_trade(session, symbol) for symbol in symbols]
        return await asyncio.gather(*tasks)


await get_trades(symbols)

Answered By - zvone