Question

Why are some of my products missing or not importing into GoDataFeed even though I see they are in the product file?

Many new items are included in our file, myfile.xml. GoDataFeed has the correct file URL in the primary source settings and all the new items are in that file when I view it in my browser. However, the new items are not being imported into GoDataFeed when downloading and checking the imported catalog data.

Answer

It appears that we may be importing a cached version of your file because you are using a proxy server (content delivery network or CDN). Here are some header examples that designate different providers.

• StackPath - X-HW
• Cloudflare -  Cache-Control or Server: cloudflare
• Fastly - Fastly-Debug-TTL

When going to your file location with one IP vs another IP the amount of items in the file changes. In one instance, we see fewer items; in another, we see more items in the file. This means we are being provided with different versions of the file.

You may want to either exclude this product file from your CDN cache or whitelist GoDataFeed's IP addresses in your Proxy Server. 

Excluding files from the CDN cache

Here are articles about excluding files from some of the popular CDN providers:

• Prevent CloudFront from caching certain files (amazon.com)
• Using HTTP Caching Headers to Exclude Assets from a CDN - KeyCDN Support
• EdgeRules: Excluding Files from CDN Cache – StackPath Help

Whitelisting GoDataFeed's IP addresses

Below are the GoDataFeed IP addresses. If you need to whitelist IPs, please whitelist the ones found below. These are also found in our help article here.

• 107.23.73.120 
• 54.197.149.152 
• 108.232.150.201
• 34.203.74.245
• 34.230.148.116
• 34.207.5.217