[FIXED] TakeFirst() in scrapy to take all data

Issue

I am really beginner and have this code:

class ActivityItem(scrapy.Item):
Id = scrapy.Field(
output_processor=TakeFirst()
)
EconomicActivityDescription = scrapy.Field(
output_processor=TakeFirst()
)

What should I do, when want to ActivityItem scrapy all, not only first not null?ň

output_processor=TakeFirst()

Solution

You can use any of the built in processors as described in the docs.

You can use Join if you want to return all the items joined by a given separator or use Identity to return the value as a list of all the items.

1. Using Join

from itemloaders.processors import Join
class ActivityItem(scrapy.Item):
  Id = scrapy.Field(output_processor=Join(","))
  EconomicActivityDescription = scrapy.Field(output_processor=Join(","))

2. Using Identity

from itemloaders.processors import Identity
class ActivityItem(scrapy.Item):
  Id = scrapy.Field(output_processor=Identity())
  EconomicActivityDescription = scrapy.Field(output_processor=Identity())

Answered By - msenior_