Skip to content
This repository was archived by the owner on Apr 4, 2025. It is now read-only.
This repository was archived by the owner on Apr 4, 2025. It is now read-only.

ng add @angular/seo #729

Description

@IgorMinar

Bug Report or Feature Request (mark with an x)

- [ ] bug report -> please search issues before submitting
- [x] feature request

Area

- [ ] devkit
- [x] schematics

Versions

v6

Desired functionality

I'd like to be able to invoke ng add @angular/seo and have the CLI make my application search engine friendly. Initially I'd like to see the following feature provided by the schematics:

  • generate robots.txt with no rules but a simple instructions/link on how to setup blacklist rules
  • generate sitemap.xml with similar documentation/links as in robots.txt
  • add code that would feature detect and conditionally load polyfills needed by the crawlers
  • setup error logging via global error handler with an option to redirect the output to a remote server
  • provide facilities to easily add and remove the noindex metadata to emulate the HTTP 404 and 500 status codes

Mention any other details that might be useful

Activity

  1. hansl commented on Apr 18, 2018

    @hansl
    Contributor

    Why not make this as part of the ng new experience directly?

    provide facilities to easily add and remove the noindex metadata to emulate the HTTP 404 and 500 status codes

    That implies a router, no? We don't have a router by default (maybe we should).

  2. clydin commented on Apr 18, 2018

    @clydin
    Member

    There are classes of applications that do not need/want to worry about SEO: internal applications, applications exclusively behind a login, etc. For those cases the files and logic could manually be removed, I suppose.

    However, some of those items (e.g., conditional polyfills) are things that could be elevated beyond SEO though.

  3. IgorMinar commented on Apr 18, 2018

    @IgorMinar
    ContributorAuthor
  4. tobi-or-not-tobi commented on Apr 19, 2018

    @tobi-or-not-tobi

    If it's only a set of relatively static files that can be generated initially, it wouldn't matter to much whether its part of ng new or done in a schematics, besides the note of @IgorMinar. But I'd imagine that the real value goes into some dynamically data that will change over time, i.e. updates to the sitemap based on the router. That would require a repeatable activity and would be used while building and maintaining the app, and then initial creation only wouldn't be enough.

  5. tobi-or-not-tobi commented on Apr 19, 2018

    @tobi-or-not-tobi

    Another aspect is that part of the generated files might need to be extended by users based on dynamic data, which are not known in the app. This is a separate flow and we should be careful do allow for such manual additions.
    We could generate a sitemap index file during the initial creation of the app (i.e. ng new --sitemap) and build specific sitemaps while the app grows (i..e ng add @angular/sitemap-routes). The later would generate specific links which are simply not available initially, and write them in a separate sitemap.xml files.

    <?xml version="1.0" encoding="UTF-8"?>
    <sitemapindex>
      <sitemap>
        <loc>http://www.we-need-a-domain.com/static-routes.xml</loc>
        [...]
      </sitemap>
    <!-- developers could add specific sitemaps to the sitemapindex file. 
      <sitemap>
        <loc>http://www.we-need-a-domain.com/products.xml</loc>
        [...]
      </sitemap>
    </sitemapindex>
    

    Developers can create other manual sub sitemap files and include them into the main sitemap (which should not be generated again).

  6. donoso-eth commented on Apr 19, 2018

    @donoso-eth

    Great Idea, as i built my site, it would had helped me, if at least, I would have known the which basics I have to take care.
    My take on sitemap.xml:
    The Sitemap is very dynamic (last-mod, frequency and prority) for each URL, therefore for sitemaps we could ask what is worth to offer to the devellopper?

    1. Create an empty sitemap.xml with xml structure
    2. Based upon a json config and a string[] with routes create the sitemap.xml
      config= {
      "host":"www.mywebsite.com"
      "frecuency":"weekfly",
      .......
      }
    3. Following the idea from @tobi-or-not-tobi of @schematics/routes we could develop a small crawler that on request craw all internal "routerLink" attributes and create a list of "urls", which merged with a config file will create the string[] with the urls.

    Probably the trade off effort/results is to go for the number 2.
    The number 3 is the interesting one, for ssr I can imagine how could it be done, for client side rendering I dont´t have a clue so far....

  7. CharlBest commented on Apr 19, 2018

    @CharlBest

    I think this is a really cool idea and didn't know about any of this so adding this with one command potentially with some granularity/options would be awesome!

  8. kylecordes commented on Apr 19, 2018

    @kylecordes

    It seems to me that in the general case, for a dynamic application, sitemap.xml would be generated by server-side code that accesses the same data as the Angular application.

    Therefore - perhaps the sitemap idea fits in well with Universal, but not with client-only Angular?

    Universal is becoming more popular; perhaps angular/seo ought depend on /universal.

  9. tobi-or-not-tobi commented on Apr 19, 2018

    @tobi-or-not-tobi

    @kylecordes agreed that this would make a lot of sense in universal, but as @IgorMinar pointed out at ngconf, he's in favor to see a solution in non-universal driven websites. Ideally we try to achieve the same for both in a generic way.

  10. hiepxanh commented on May 7, 2018

    @hiepxanh

    I believe SEO is the most important think. More than speed, comfort, simple code, etc....
    Without SEO, my best beautiful fastest ever website is just nothing, it just like a business without marketing...
    I think we have two problems:

    • Problem 1: Developer should create sitemap.xml and robots.txt but he should create an automation way to automatic update that file. ( when?)
    • Problem 2: How we can update that without touching that file? That is the client side problem.
  11. hiepxanh commented on May 7, 2018

    @hiepxanh

    About the price he said. I think IgorMinar is right. More money, more SEO... It is a truth and it is acceptable. Assume that your website is dynamic content (just like news, or blog or something with auto generated content).
    About current situation:

    1. People need push more money and effort to make angular SEO able.
    • money: every website with dynamic content need a server. ( but, as Loki said "we have firebase functions" 😄)
    • effort: every website with dynamic content need few setup. (our case is universal).
      if they choose that. how we can do it painlessly?
    1. if using client only. I heard that they have Rendertron but I think we can provide something better. or just integrate Rendertron to universal?
  12. BenjaminHofstetter commented on May 25, 2018

    @BenjaminHofstetter

    A great idea.... and maybe some kind of support for JSON lD (https://json-ld.org/)

  13. donoso-eth commented on Jun 3, 2018

    @donoso-eth

    @IgorMinar diving into the angular.io index.html to see which best practies are you using.
    I came across following script // Dynamically, pre-emptively, addnoindex, which will be removed when the doc is ready and valid tag = document.createElement('meta'); tag.name = 'robots'; tag.content = 'noindex'; document.head.appendChild(tag); which as it is written add a noindex tag whih will be removed once the app is bootstrapped.
    My question is then, for search bots other than googlebot, is this tactic not penalyzing the index as these bots will only see the tag noindex?
    Thanks for your awesome job and contribution

  14. alexeagle commented on Sep 4, 2018

    @alexeagle
    Contributor

    This issue was moved to angular/angular-cli#12120

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions