[FIXED] Splash API/lua error: attempt to index local element (a nil value)

Issue

I'm writing a lua script that I want to use with scrapy + splash for a website. I want to write a script that enters a text and then clicks on a button. I have the following code:

function main(splash)
   local url = splash.args.url
   assert(splash:go(url))
   assert(splash:wait(5))

   local element = splash:select('.input_29SQWm')
   assert(element:send_text("Wall Street, New York"))
   assert(splash:send_keys("<Return>"))
   assert(splash:wait(5))

   return {
     html = splash:html(),
   }
end

Right now I'm using the splash API to test if my code runs properly. When I click "Render!" I get the following message:

{
  "info": {
      "message": "Lua error: [string \"function main(splash)\r...\"]:7: attempt to index local 'element' (a nil value)",
      "type": "LUA_ERROR",
      "error": "attempt to index local 'element' (a nil value)",
      "source": "[string \"function main(splash)\r...\"]",
      "line_number": 7
  },
  "error": 400,
  "type": "ScriptError",
  "description": "Error happened while executing Lua script"
}

So for some reason element is still nil when I try to send "Wall street, New York". I don't understand why; if I enter the following in chrome console:

$('.input_29SQWm')

I find the desired element!

Q: Does anyone know what I'm doing wrong?

Thanks in advance!

Solution

As the error message tells you you try to index a the local 'element' which is nil. The error occurs in line 7: assert(element:send_text("Wall Street, New York"))

So why is it nil? In line 6 we asign a value to element

local element = splash:select('.input_29SQWm')

Obviously splash:select('.input_29SQWm') returns nil

Let's have a look into the documentation:

http://splash.readthedocs.io/en/stable/scripting-ref.html#splash-select

If the element cannot be found using the specified selector nil will be returned. If your selector is not a valid CSS selector an error will be raised.

Your mistake is not handling the case that select might return nil. You cannot blindly index a value that might be nil. Also you should use protected calls when calling functions that raise errors.

Now it is up to you to find out why select did not find an element using that selector.

I recommend you read something about error handling in Lua befor you continue.

Answered By - Piglet