[FIXED] Asyncio unexpected condition race problem

Issue

I have a weird problem using asyncio and aiohttp. I may be experiencing race condition. I am running this async function cvim_vlan_list twice from loop with different x variable. But what I am experiencing is that token I get from first await within the function is somehow rewritten in the middle of the code - the means cvim_headers object is the same which shouldnt be. I put comments next to the code. I have really no clue why the value is rewritten.

Initializing loop

        self.loop = asyncio.new_event_loop()
        checks_to_run = self.returnChecksBasedOnInputs()
        self.loop.run_until_complete(asyncio.wait_for(self.run_all_checks_async(checks_to_run),180))
        asyncio.set_event_loop(self.loop)

In the file where I am passing loop

x = 0
cvim_tasks = []
for cvim_scan_container in self.constants.cvim_scan_containers:
    cvim_tasks.append(self.loop.create_task(self.cvim_vlan_list(cvim_scan_container,x)))
    x += 1
        
await asyncio.gather(*cvim_tasks)



async def cvim_vlan_list(self, cvim_scan_container, x):
        cvim_vlans = []
        network_ids = []
        # 2. Get cvim_container_token for each az
        cvim_headers = self.constants.cvim_headers
        cvimAuthenticationData="{\"auth\": {\"methods\": [\"credentials\"], \"credentials\": {\"username\": \"calipso\",\"password\": \"" + self.constants.azpasswords[x] + "\"}}}\r\n"
        response = await apiCaller().callAndReturnJson("POST",f"https://{cvim_scan_container}/auth/tokens",headers=cvim_headers, session=self.session,payload=cvimAuthenticationData,log=self.log)
        cvim_container_token = response['token']

        #update headers with token
        cvim_headers['X-Auth-token'] = cvim_container_token  ############## HERE ARE cvim_headers different
        # 3. RUN SCAN in each az
        scanData="{\"env_name\" : \"cvim-fci-pod"+str(x+1)+"-muc-eip\",\r\n\"log_level\": \"warning\"\r\n}"
        response = await apiCaller().callAndReturnJson("POST",f"https://{cvim_scan_container}/scans",headers=cvim_headers, session=self.session,payload=scanData,log=self.log)
        scan_id = response['id']

        ############## HERE ARE cvim_headers the same
        ############## The token from second def execution is also in the first def execution, that means cvim_headers are same which results to error from API 
        # 4. Periodically check if scan is completed
        isScanRunning = True
        while isScanRunning == True:
            self.log.info('checking scan')
            
            self.log.info(cvim_headers) ########  HERE THE token from second def execution is also in the first def execution, that means cvim_headers are same which results to error from API 
            response = await apiCaller().callAndReturnJson("GET",f"https://{cvim_scan_container}/scans?env_name=cvim-fci-pod"+str(x+1)+"-muc-eip&id="+scan_id,headers=cvim_headers, session=self.session,payload="",log=self.log)
            self.log.info(response)
            if response['status'] == 'completed':
                self.log.info('Scan completed')
                isScanRunning = False
            else:
                await asyncio.sleep(5)

Solution

You are using something called self.constants.cvim_headers. Presumably this is a dictionary (containing some default headers). In cvim_vlan_list you do this with it

cvim_headers = self.constants.cvim_headers

and then later on you update it with

cvim_container_token = response['token']
cvim_headers['X-Auth-token'] = cvim_container_token

I think the problem you are encountering is just a misunderstanding about what's happening here. The assignment doesn't put a copy of self.constants.cvim_headers into cvim_headers, but rather it places a reference to the dictionary referenced by self.constants.cvim_headers into cvim_headers. Each time you assign to the X-Auth-token key, you are changing the dictionary referenced by both cvim_headers and self.constants.cvim_headers, overwriting the previous value associated with that key.

To prevent this, you need to create a copy of the default headers and assign that instead.

cvim_headers = self.constants.cvim_headers.copy()

Answered By - dirn