Scrapy reptile added when redis support, pits occur:
- When the request object passed in Scrapy, are passed by reference, that is modified in one of the request, it will change in other places; and replaced after redis, this will become the value passed; the reason? redis request object from the database over deserialized;
- About disabled duplacte filter; the original reptile scrapy in, you can set a variable; but in scrapy redis, the need to redefine themselves a return type of false, otherwise it will error, see detailed reasons being given available;
This time using python thoughts
- For simple python deal with the problem, really convenient, but really with it, there will be a lot of knowledge to learn, is not a simple tutorial python can handle; perhaps the lack of knowledge, python still not stable, for example, python interpreter direct collapse?