User-agent: * Allow: /index.html? Disallow: /assets/* User-agent: Twitterbot Allow: /assets/img Allow: /assets/vid User-agent: facebookexternalhit Allow: /assets/img Allow: /assets/vid