Byline Appeal Made Easy

Byline appeal

JOHN MUELLER: All right, welcome everyone to today's Google SEO Office Hours Hangout. My name is John Mueller. I'm a Search Advocate at Google. And part of what I do are these Office Hours Hangouts, where people can jump in and ask their questions around Search. A bunch of things were submitted on YouTube already so we can go through some of that. But maybe first of all, I just wanted to wish you all happy new year. Thanks for sticking around last year through all of the, I don't know, ups and downs and sideways and everything. Hopefully this year will be a little bit easier, but it's starting off with a bang. So we'll see. Hopefully, at least, with these Office Hour Hangouts, we have something kind of regular that's not too, too crazy. Anyway, so we can jump through some of the questions that were submitted. But maybe first of all, if any of you want to ask a question, something on your mind, feel free to jump on in now. No one. Oh, I see someone raise their hand. Go for it. BILAL AHMED: I don't-- if you can hear me-- JOHN MUELLER: Yes. BILAL AHMED: OK, perfect. So I'm trying to index a site that has about a million products. So I'm trying to index it slowly by just submitting the products first and then categories, pages, rather indexing, tags, and everything all at once. So this is being done from last few weeks. And I've managed to index about three [INAUDIBLE] products. So is there something I can do extra to expedite the process so I can get indexed those products quickly? JOHN MUELLER: Not really. I assume this is a new website. BILAL AHMED: Yeah, it was actually published three, four months ago, yeah. JOHN MUELLER: Yeah, OK. So usually what happens on our side is, we try to index a bunch of content on especially new websites. But we're a little bit trying to be almost on the safe side when it comes to that kind of content. So in particular, it takes a lot of storage, it takes a lot of time to index really large websites. So we need to make sure that it's really worthwhile. So one of the things that we do is, we might-- in the beginning, with a new website, we might be a little bit more conservative and a little bit-- hold back a little bit on the indexing speed, on the crawling speed so that we don't cause problems on the server. And over time, as we see that this is a really good website, that it kind of embedded well within the rest of the web, and we'll pick up again and crawl a little bit more. So what you're seeing there is essentially normal and expected. And it's not that there is a simple way to just say, OK, Google, index a million pages. But rather you have to kind of show to Google that, actually, the millions of pages that you have on your site, they're worthwhile and they're important to the index, because they contain something that people are looking for, where people are actively going to your site, recommending it to other people. BILAL AHMED: So that was really interesting. Actually, we have noticed the quality and the-- from your guidelines, et cetera. So this is actually a parts website, mechanical parts. It has a specific niche or industry that is searching for those parts. So I believe it might take much longer than the usual website. What do you think, yeah? JOHN MUELLER: I don't know. It depends. It's possible. So one of the strategies that some sites use, especially in the beginning as they're building things up, is to concentrate on fewer pages that are really important for you, and build up on those pages and expand from there. So with a parts website, one of the things you could do is focus, for example, on the categories of parts that you have. And instead of focusing on all of the millions of individual parts on the categories, because then people will still be able to find your categories if they search specifically for the part number or the name of the part, but you'll have a lot fewer pages that you want to have indexed in the beginning. And we can focus on that first. And then over time, you can say, oh, well, these parts are also good to be indexed, and these parts, so kind of controlling indexing a little bit to match what you actually want to have found. BILAL AHMED: Perfect, yeah, I got it. Thank you so much. JOHN MUELLER: Sure. It's always a bit challenging with a new website, and if you have a lot of content, and especially if it's something where you'd say, well, it's normal content. It's not that you're automatically generating content in the millions of pages. But it takes time. And kind of learning to focus first and then expand from there, I think that really helps. BILAL AHMED: Thanks. SAKET GUPTA: Can I ask the next one? JOHN MUELLER: Sure, go for it. SAKET GUPTA: OK, hello, John. So my question is regarding links reporting Search Console. So as of now, we cannot see all the links, but regarding some links which we are visible on the report. So are Google filtering based on some reference tag-- I"m sorry, I forgot the actual name. JOHN MUELLER: Nofollow? SAKET GUPTA: Yeah, the nofollow, sponsored, or UGC and all. So is Google shows all the links, all type of links, or Google filters some type of links or attribution we can say? So just wanted to know. JOHN MUELLER: We don't filter by the type of links, which is sometimes a bit confusing. But what we do there is we show a sample of the total links. It's not that we would show all of them there. It's not that we would filter by nofollow or by disavowed links or things like that. It's really just the sample. And we try to make it a representative sample. But sometimes you have a little bit more and sometimes you have a little bit less. SAKET GUPTA: Thanks. NEERAI PANDEY: John, I have followup on this. JOHN MUELLER: Sure. NEERAI PANDEY: So John, in case there is any site migration, on that case, can we expect immediately moving links from one property to next property, or it will take time for Search Console to start showing links on new property? Because we are noticing that, in some websites, links reported is not fetching for new property. JOHN MUELLER: I think it would be normal that it takes a bit of time with a site migration to update all of the data in Search Console. I don't know if there's anything different with regards to the links reports there. But it is pretty normal, when you do a site migration, that the data takes a bit of time. Specifically with regards to links, what happens there is we track the link based on the canonical URLs on both ends. So on the one hand, the page that it's linking from, we take the canonical URL from there and the canonical URL of the page that it's linking to. And if you do a site migration, then individually, the pages on your website will ideally shift a canonical to your new site. And then when that canonical shifts, then we count that link as being from the original source to the new canonical URL. So that's something where, theoretically, if we just don't do anything special for site migrations, then that's a process which will take a bit of time, on the one hand for us to shift the canonical, on the other hand, for us to shift the data in the links report. So I don't know if we do anything special for site migration cases where you have the setting in Search Console. But I would expect that this takes several months, maybe, to settle down to the new domain when you're doing a migration like that. And it's probably similar for the other reports. But especially with the links report, it's really something where we really have to focus on the canonicals on both ends. And that especially takes a little bit longer. NEERAI PANDEY: So in that case, if new property is getting indexed, then immediately the links from old property will be shifted, or Google needs to go and crawl those links again, and then only it will map canonical and then shift? Meaning, it will depend on links source crawling? JOHN MUELLER: No. NEERAI PANDEY: Or it will depend only-- OK. JOHN MUELLER: No, it's based on the destination. So it is a little bit, I don't know, complicated, in the sense that, for the links, we have those both ends, like the page it's linking from and the page that it's linking to. And if either of those changes the canonical, then we will update that in our internal links. But especially with the links report, like so many steps that lead to the data being shown in the links support, I could imagine it just takes a little bit longer. NEERAI PANDEY: OK. Thanks. JOHN MUELLER: Sure. SAKET GUPTA: One other question I just wanted ask, and then [INAUDIBLE]. So suppose a website receives a manual action regarding unnatural links. So how of a master or a website property owner will analyze those links, OK, because of those links, I'm receiving the manual action. Because in manual actions, there is no example links are given. So it's very hard to identify which link's causing the manual action. JOHN MUELLER: Yeah. So I think the general theme is kind of like, well, if you don't show all of the links, how can I fix all of the links? And in general, when it comes to manual actions, it's not something that is based on individual links. It's really based on a broader pattern. And that's essentially the broader pattern that you should look for and try to resolve, and not those individual five links there and five links here kind of thing. So for manual actions, I think in pretty much all cases, it would be the situation that, in the length report, you see enough information to recognize that broader trend of a problem. And based on that, you can fix that broader problem. And the manual actions team is, for the most part, not going to be picky in that, oh, you fixed 700 links, but you didn't fix these two links. But rather, they want to see that you understood what the problem was and that you were able to resolve that. So that's kind of the general idea there. And for that, you don't need to have all of the links. You need to kind of know what the problem is. And most of the time, you kind of know what you or what your site did to get that manual action. And then fixing that is something that needs to be done. One of the things, especially I think with the links-based manual actions, is it takes a lot of manual work on our side, also, to review those. So sometimes they just take a little bit longer to be reviewed, which means, from a practical point of view, it makes sense to really go through and try to clean up as much as possible with regards to those links. So that's something to-- if you do have a manual action based on links, then finding the broader pattern is really important. But also really being clear that you recognize it and cleaned it up is really important. SAKET GUPTA: Again, as you said, the broader pattern-- so recently in a couple of months, we saw links are coming from Google websites itself-- for example, hotel websites, Google Hotels. So from Google Hotel website we are getting hundreds of thousands of links for a specific single page without nofollow, and the anchor text is remain the same. So will this cause or maybe the reason for the manual action? JOHN MUELLER: No, no. That should not be the case. If it's a nofollow link, definitely not. If it's something which is a natural link that is there, then also definitely not. It's really more a matter of, maybe you reached out to 1,000 bloggers and tried to get a link from their site. Or maybe there is some other pattern involved where you're paying people for links or you're doing some kind of a link exchange kind of scheme, then that's the kind of thing where the [INAUDIBLE] team would step in and say, OK, we need to take manual action here. For nofollow links, we can ignore those automatically. That's perfectly fine. Just because it's a large number of links, that doesn't matter. If it's a similar anchor text, that doesn't matter. It's really the thought behind the links that's important for us. SAKET GUPTA: OK, thank you, John. NEERAI PANDEY: John, regarding nofollow, we all know that they don't need to be disavowed. But what about UGC and sponsored links? JOHN MUELLER: You also don't need to disavow those. SAKET GUPTA: Also the links usually are tagged as reference. JOHN MUELLER: Which one do you mean? SAKET GUPTA: A reference. JOHN MUELLER: Referrals, like-- SAKET GUPTA: Yes. JOHN MUELLER: How do you mean? SAKET GUPTA: I don't know. The links [INAUDIBLE] NEERAI PANDEY: Referral is probably, I think, Google [INAUDIBLE] attribute which is ignored by Google crawler. JOHN MUELLER: Yeah. NEERAI PANDEY: This is what I know about. Google Crawler ignores referral link. That is also equivalent to a dofollow link. SAKET GUPTA: Yes. And those links are visible in the links report. So do we need to disavow those links or not? [INAUDIBLE] JOHN MUELLER: No. I mean, if they're flagged as such that they wouldn't be passing page rank, like with a nofollow, UGC, sponsored, then that's perfectly fine. You don't need to disavow those. ROBB YOUNG: And John, that stuff's still really only worth doing with a manual penalty and not just as a general tidying up? JOHN MUELLER: Yeah. Yeah. The one case I might do that outside of a manual penalty if you really know that someone working on your website did something weird in that direction, and you just want to make sure that it doesn't end up in a manual action. But for normal websites, on a day-to-day basis, you don't need to do anything with the disavowed report. NEERAI PANDEY: John, regarding this, I had one idea. So when Google already discounts those links when manual action happens, and we have seen a lot of times webmasters disavow other types of links which are not worth to be disavowed. And you also suggested that it is not the web spam team 100% expects links to be disavowed. Then why Google wants webmasters to disavow those links or do some exercise because Google is already just discounting those links? And the chances are, sometimes webmaster make more damage on the website. JOHN MUELLER: Yeah, I think at some point we'll get into the position that we can really completely handle this. But on the one hand, it's a way for people to clean things up if they are aware of issues. And the other part that I think always comes up a little bit when it comes to links is the general fear that a competitor might be harming your website. And that's something where I think we do a really good job of recognizing those situations and ignoring them. But I know it's easy to lose sleep over something like that because it feels like something that you have no control over. And by using the disavow links report, that's something where, if you do recognize something where a competitor is doing something really weird with regards to links to your website, then you can go in there and say, OK, I just want to be 100% sure that this is not something that Google will ever consider as something that I'm doing. So kind of that peace of mind aspect, I think, is also quite useful there. But I think it would be nice at some point to be able to move away from just focusing on links so much, but probably not soon. ROBB YOUNG: Also the other option is to just not allow users to do it, in which case they're screwed permanently. JOHN MUELLER: Yeah, I-- ROBB YOUNG: You're asking Google to do more work to save you work, which for the most part you don't need to do if you haven't done anything wrong anyway. So it's self-policing in a lot of ways, other than genuine attacks on other sites. But they're few and far between, the genuine ones. Or I should be more specific-- the genuine ones that work. I mean, it goes on all the time. Our site has got a ton of junk links coming into it. But we didn't build them, so we just ignore them. But genuinely malicious, effective negative SEO attacks are-- you just can't find them. I don't work for Google by the way. OK, I'm going now. JOHN MUELLER: Thank you, Robb. OK, let me run through some of the submitted questions. And we'll probably have more time for other things that come up over the course as well. I have five brands that each cover different non-overlapping territories, but all brands offer the same service. I have a website for each brand. The content for each website is the same, other than the territory that's covered. What's the best approach for SEO for website content so that one brand doesn't penalize the other brand with duplicate content? Is it to write different content for each site, or to use JSON LD to tell Google the target territory for the content, or something else? So it sounds like the content is in the same language, which is something that I think would make it very different. Because if the content were just translated into different languages, then it would be unique content. We wouldn't have to worry about duplicate content. So that's one part that sometimes comes up. But it sounds like maybe in this case, everything is in English and for different countries that speak English, for example. One thing you can do here is use the hreflang attribute to tell us about these different versions and say, this is the English version for the UK, this is the English version for the US, that kind of thing. That helps us. What happens there with the hreflang is that we will index these different versions in most cases. And when a user searches, we'll try to show the version that is most suited to their location. So that's one approach that you could do there. The other approach is, like you mentioned, making sure that the content is slightly different. A third approach might also be to say, well, if this content is really the same everywhere, then maybe it makes sense to concentrate it into one site. Usually, I recommend trying to reduce the amount of content and focusing on fewer versions rather than more versions, just because it makes that single version, if you have a single version, much stronger, in that we can understand that all of the value of your site is concentrated there. And when someone searches for something general, we can say, well, this is a really strong and good website for this topic. We can show this one. Sometimes that's not feasible when you're targeting different countries or different territories, sometimes for policy reasons or, I don't know, some other reasons. So if you need to have different versions then you kind of have those options of making different content or using hreflang attributes. The other thing may be worth mentioning here is that there is no penalty for duplicate content. It's not that we will say your website is lower quality or spammy or problematic if you have the same content across different websites for different audiences. What happens on a practical level is, if it's exactly the same page, then we might just index one version of that page, and we'll try to concentrate the value there. Because we think it's exactly the same thing, so we put it all into one page. What can also happen if a part of the page is the same is we'll index the individual versions. And if someone is searching for something that is within this chunk of text that is shared across these pages, we'll just pick one of those pages to show in the search results and we'll kind of filter out the other ones. And usually we'll try to pick the one that best matches what the user is looking for. But sometimes that's kind of tricky if we don't have enough information about what exactly the user is looking for. So those are kind of the different approaches there. And I think there are pros and cons to all of those approaches. So it's not that there is one solution that will always work if you have five brands and you want to keep them separate. But different approaches might make sense in your particular case. My website is an e-commerce site and my website's index pages have been declining. What can I do to increase my website's index number? So I think we talked about this in the beginning slightly, as well, with regards to the site that had millions of pages and is just getting started. In particular, when it comes to indexing, we don't guarantee that we will index all pages of the website. And especially for larger websites, it's really normal that we don't index everything. And that can be the case that maybe we just index 1/10 of a website because it's a really large website and we don't really know if it's worthwhile to index the rest. Sometimes we index 90% and you're kind of struggle with those remaining 10%. But that balance between trying to index as much as possible and trying to focus on the content that we think is actually going to be shown in Search, that's something we always have to struggle with. So usually, the approach that I recommend is kind of twofold. And we talked about this in the beginning, as well, on the one hand trying to reduce the number of pages that you want to have indexed so that you can focus a little bit more on fewer pages and making those stronger. That's one approach I think makes sense and is, depending on the website, kind of possible. The other approach is to really significantly work to improve the quality of your website overall and to make it so that Google and users and everyone understands that this is a really important website and it would be a flaw in Google's algorithm if Google's algorithms decided not to index as much as possible from this website. So that's kind of the other approach there. Usually it's a little bit easier to focus on fewer pages and just making those stronger than to significantly improve the quality of a whole website overall, especially when you're talking about a really large e-commerce website. But sometimes it's worthwhile to take that time to invest and try to significantly step things up. Share some brief information on SSR and CSR and how it impacts indexing and rankings. So SSR is server side rendering and CSR is client side rendering. So usually, those are the aspects of how using JavaScript on a website with regards to showing the content on your website. And the general approach there is, if you have a JavaScript-based website, if you don't do anything special, then that would be called client side rendering, because in your browser it has to process the JavaScript and then render the pages. Whereas if you do something special on your website with that JavaScript-based website so that this server actually processes the JavaScript and then sends the HTML code to the browser, then that would be called server side rendering. And there are pros and cons to both of those approaches. Sometimes there is a speed aspect involved there, as well, and that server side rendering takes a bit of time. But it makes things a little bit faster for users, so you kind of have to balance things out there. I would recommend checking in with Martin. Martin also does Office Hours for SEO, for JavaScript SEO in particular, and trying to ask him I guess questions more specific to what exactly you're looking for. Because this is a really broad topic and I don't think there's like this one thing where it's like, oh, you should do exactly this and it will work well. But rather, usually you have some existing framework on your website that's based on JavaScript and you need to figure out what you need to do there. So I'd check in with Martin on that. How do AMP pages help in ranking? Also, how does it make sense when we have a PWA client side rendered, light, quick-loading pages, and we also make AMP for the same pages? So AMP pages do not affect ranking. That's kind of the first thing first off. So it's not that you need to use AMP if you want to rank well in Search. I believe there are still some features that rely on the AMP framework with regard to how we show the content. There is also Web Stories, which is based on AMP. So if you want to use anything around that, then maybe it makes sense to use AMP. But if you're just worried about speed, AMP is a great way to make really fast websites. But it's not the only way to make really fast websites. So that's something where if you do have a really fast website already, then maybe that's already fine. So yeah, I don't know. I don't want to talk you out of using AMP because I think it's fantastic to have a framework that makes it easy to make really fast websites. But at the same time, you don't need to kind of like turn everything around, and if you have a really fast website already then change your framework and change all of that just so you can use AMP. Regarding the upcoming mobile first indexing update, if content is similar and crawlable for Google bot for mobile and desktop, but the mobile version is not responsive, then how can this update affect those websites? I see several blogs claim that responsive web design is a first step for this, but I don't see any information from the Webmaster Central blog for this claim. So I'm not 100% sure what exactly you're asking here. But if you have a separate mobile and a separate desktop version, and you have those versions linked properly so that we can understand the connection between those two, then that's essentially what we need so that we can understand that those versions are there. When it comes to mobile first indexing, we will only index the content from the mobile version, though. So that's something where, if like you said there, the content is similar and crawlable for mobile and desktop, then that's kind of what we would focus on. And we would use the desktop version as a way of understanding, oh, there's a desktop URL for this page as well. And if someone on desktop is searching, we can show them the desktop URL. But otherwise, we'll focus indexing on the mobile version. And it's not the case that the mobile version has to be responsive design. If you have separate mobile URLs, that's perfectly fine. I don't think it's the optimal approach to have a separate mobile URL. But if that's the way your website is set up, then if the mobile URL has all of the content, then that's essentially OK. SAIDUL HOQUE: Hi, John. JOHN MUELLER: Hi. SAIDUL HOQUE: May I ask a question? JOHN MUELLER: Sure. SAIDUL HOQUE: So my question is about targeting different regions in a country. So one of my clients, they provide the same service, but for each state, they have different branding. And they have different support website for each brand. Now they want to use the same content on each website. Do you think this will be a problem for the duplicate content issue, and if we have this issue, then how we can rectify this problem? JOHN MUELLER: I think we just went through that question briefly right before you joined. So I'd check out the recording afterwards and see if that answers your question, because that's pretty much exactly the same question there, like the same content for different brands kind of thing. SAIDUL HOQUE: Thank you. JOHN MUELLER: Sure. [INAUDIBLE],, I think you had some more details on the question there. Do they have a microphone that maybe you can ask there? NEERAI PANDEY: John, I think he's talking more about [INAUDIBLE]. Is it really required to have responsive page if the content is similar in mobile and desktop version? JOHN MUELLER: OK, so the viewport metatag I think is specific more to the responsive design setup. So that's something where if you have a separate mobile URL and a separate desktop URL, that's perfectly fine. For indexing, we focus on the content of the page. We don't see if it's mobile friendly. We essentially just use a mobile version for indexing. Mobile friendly is something that we use for the page experience ranking factor, which is coming in May. So it makes sense to have a mobile friendly page. But if you have a mobile version and a desktop version and the mobile version is mobile friendly, then that's perfectly fine. How would you remove date snippets appearing in the search results for static pages, like home, services, contact, et cetera. So we don't have a metatag to remove data in particular from pages that we show in the search results. Usually, we try to understand when it makes sense to show a date on a page. And we'll try to show that in the snippet of the search results itself. If there's a specific date that you want to have shown, then you can use, I think, the date structured data in the article structured data type, I think, where you can tell us which date we should use there. So if you do have a date, you can tell us there. If you don't tell us there, then we'll look at the content itself to try to find a date within that. If you don't want a date to be shown, then it's not possible to suppress that date snippet from being shown. But what you could do is, of course, make sure that there is no date shown on your page. So if you don't have any dates in the HTML page, then we don't really have much to pick up on to show there. So especially when you're talking about things like a home page or a contact page, for the most part, you probably don't have dates on there anyway. So that should be something where you shouldn't see that too frequently. If you do see this showing up on pages where you say, well, it's a normal Contact Us page, why is Google ever showing any dates there, then I would love to have examples of that. So in particular, a query where you're seeing that happen and the URLs from your site where that's happening from, so that's something that we can take up with the team here that works on showing dates and recognizing dates on a page, where we can say, well, maybe we're recognizing a phone number as a date accidentally. And we need to be able to fix that. After the disavow tool is used, does a domain carry any mark that it may hold it back? No, no, the disavow tool is purely a technical thing, essentially, that tells our systems to ignore these links. It's not an admission of guilt or any kind of bad thing to use a disavow tool. It's not the case that we would say, well, they're using the disavow tool, therefore they must be buying links. It's really just a way to say, well, I don't want these links to be taken into account. And sometimes that's for things that you have done or someone working on your website has done in the past. Sometimes that's for things that you just don't want Google to take into account for whatever reason. And both of those things are good situations, right? It's like you recognize there's a problem and this is a tool that you can use to resolve that. And that's not a bad thing. So it's not the case that there is any kind of a red mark or any kind of flag that's passed on just because a website has used the disavow tool. How can a bad SEO practice rank first and a good practice website ranks second and third when the implementation of the new Google updates hits big sites or it's never going to happen? I think this is something that comes up pretty much all the time. It's like, I'm ranking below a website and they're doing one thing bad, and why doesn't Google notice that and just remove that website completely from Search, because they're keyword stuffing or they've bought links or they've done some other crazy thing? And I don't think that this is something that is, for the most part, useful to focus on as a site owner. Because essentially, you're focusing on someone else's website and trying to find problems in that other website rather than taking the time that you have and focusing on your website instead. So that's kind of the first approach that I have there. The other thing is that our algorithms use so many different factors when it comes to ranking that it is very common that a website does not do everything perfectly. I think that's the most common situation overall, in that a website will do some things really well and some things kind of OK. And maybe we'll do some things really badly. And because of that, it's not the case that we would say, well, we'll just remove all of the websites that aren't perfect, because I think the Search results will be pretty empty. And instead our algorithms are built in a way to try to find the overall view of that website. So that could be simplified into, well, we take the average of how good the website is. It could also be that, for some things, we can recognize that a website is doing something badly and we try to ignore that. And I think that ignoring option is really important and really a strong part of our algorithms, because that means that, even if you follow bad advice from the internet somewhere, it's not that your website will automatically be discarded and never shown in Search, but rather, we recognize, oh, you're using keyword stuffing on your pages. And we can just ignore that keyword stuffing and we'll focus on the good parts of your page. So if you get weird advice from friends or from the internet about your website and you follow that advice, and we can recognize that you're trying to do something sneaky there, then we'll try to ignore that and instead focus on the good parts of your website. So that's kind of a good thing for your situation. But obviously, if your competitor is doing something wrong and we're just ignoring that bad part, then that can be a bit frustrating. So instead of focusing on the things that your competitor is doing wrong, try to find ways that you can improve your website that are more, I'd say, sustainable for the long run for your website in particular. And that could be to say, well, the competitor is doing these things badly and maybe Google is ignoring those, but I can do those things really well so that when Google re-evaluates my website over time, it'll see, well, actually, I have a lot more things that are lined up and done well. So that's definitely one approach. The other thing to keep in mind is that, a lot of times, these situations are based on kind of technical elements of a website, where if you look at a website and you say, oh, their HTML is not good, or they have maybe this specific score in a testing tool that you have and your website has a better score, those technical elements do matter to some extent. But what is also really much more important almost is the overall quality and the value that your website brings to the web. So that's something where it's very easy, as an SEO who focuses on tools and numbers to say, well, technically my website is better. But practically, maybe that other website is just a lot better, in that it provides a lot more value to users. It works really well for those users. So they might be doing things like, I don't know, using frames on the website and doing these old school HTML things that are not really great. But the value that they provide is just so much more than kind of this technically sleek website that maybe you have at the moment. So focusing on kind of the areas of improvement where you see you can surpass those competitors is important, but also making sure that you're providing something that is really significantly better than them overall, and not just from a technical level, but kind of just purely from a user level as well. And I think evaluating that non-technical aspect is really hard. But one of the things that I find really insightful every time I look at it is when you do a user study. So we did a blog post specifically, I think, for the core updates and for the panda update at a time with lots of questions you can ask yourself or you can ask users about your website. And that's the kind of thing that I would take and do a user study, and maybe find 10, 20 users of your website and go through these questions in an objective way to really get input on where you can improve your website, not purely from a technical level, but also kind of from a user level as well. Let me see. Maybe we can run through some of the other questions here. The large website with millions of products, I think we talked about that briefly, the most important ranking factors. I don't think we list the most important ranking factors, so I'll have to disappoint you there. If I submit a blog post today and add more content in 10 days from now, do I need to submit an updated sitemap for the blog post to get updated or will it just get recrawled? So the sitemap file definitely helps us understand when there is new and updated content. So that's a great thing you can do. If this is something that is-- if you're using a CMS like WordPress, for example, then this will be completely automated. There's nothing that you need to do there. If you're using a setup where you don't have a kind of an automatic sitemapf file, then you can use the Submit to Indexing Feature in Inspect URL and let us know that way. From a practical point of view, if you're doing anything more than a website with, let's say, 10 pages, I would strongly recommend using a sitemap file and automating this rather than doing it then. Should a 404 page contain a self-canonical? Is it OK if I added one and then removed it? We don't look at any content on a 404 page. So if a page is returning a 404 status code, then essentially we say, OK, that's fine, that's enough for us. So if there's a canonical there, if there's a no index there, if there is anything else on that page, we essentially ignore that. If it's a 404 page, it's a 404 page. Cool. OK, I think we ran through those questions. What else is on your mind? What else can I help with? MICHAEL LEWITTES: John, actually, that question about a date popping up actually reminded me of something. A friend of mine asked me a question about our site. And I thought it was a pretty good question because it had, well, wide appeal for others. And it's about different types of content on one site. So she has a website that has resources for women who are interested in stand-up comedy, where you can take classes, how to write a routine, that kind of stuff. It's very evergreen. But she also has a blog that's written by established comedians tied to news events. So she asked me if the newsier articles written by these well-known comediennes, or experts, if you will, should they have bylines and timestamps, while the evergreen content does not? So I guess that-- JOHN MUELLER: I think that makes sense, yeah, to have something like that. It's sometimes, I imagine, a bit tricky if you mix those two on the same site and if it's hard for us to recognize which part is the newsier content, which part is the evergreen content. So something like separating that out in a separate URL structure I think would make sense. But in general, if you have something that is timely, then I would certainly go for making sure that you have that byline there, that you have to date on the page, that you have the date in the structured data as well, just so that we can clearly understand this is something new that just came out. Maybe we can show it in Discover. Maybe we can show it otherwise in the search results. MICHAEL LEWITTES: Thanks. PRAVEENKUMAR ELANGOVAN: Hey, John. JOHN MUELLER: Hi. PRAVEENKUMAR ELANGOVAN: Hi. I'm having a question on X-Robots-Tag. So we [? aren't ?] supposed to use a private folder on the page level noindex metatag. So I just go through the X-Robots-Tag tag. So is there a permanent solution preventing the Google from the index, or it's like something similar to robots.txt blogging? So we need to blog the page and delete permanently from the Google or any other search engines. So can we use the X-Robots-Tag? JOHN MUELLER: Sure, yeah. I don't know about other search engines. So in particular, the X-Robots-Tag is kind of a special variation of the robots metatag. So I don't know if all search engines process that exactly the same way. But if you're serving the X-Robots-Tag on a sitewide level for all of the URLs from your website, then that's a good way to prevent them from being indexed. PRAVEENKUMAR ELANGOVAN: So the server level configuration or the header.php, which method do you prefer for the [INAUDIBLE] blogging? JOHN MUELLER: Totally up to you. We only see the end result. And how you implement that end result is totally up to you. PRAVEENKUMAR ELANGOVAN: OK, thank you. JOHN MUELLER: Sure. BILAL AHMED: John? I have a question actually. There's a website, which was put to noindex through metatags and was deindexed 10 days ago, when we realized and we removed that noindex property or tag. And we're just waiting for about 10 to 12 days now to get crawled by the Google. Is this something it'll take extra long when this happens usually? Because the average we get indexed our sites in a few days, like three to four days or maybe seven days. But it's taking a bit longer. Anything you can add to it? JOHN MUELLER: Is it the whole website or is it just pages within the website? BILAL AHMED: Actually, I was reviewing-- it was the whole website, which was put to noindex. Eventually, the whole website was removed from the Google. So we looked and we find that the whole website is noindex. So we just-- yeah. JOHN MUELLER: And how long ago did you remove the noindex? BILAL AHMED: It was about 26th and 27th of December. Now it is 8, so it's quite long, yeah. JOHN MUELLER: Now, so usually for the most part, I think there are two aspects that play into that. On the one hand, for most websites, we try to recrawl at least some of the pages every couple of days. So even if the website was on noindex for a long time, then we should try to recrawl those every couple of days. So it's like end of December, and now it feels like, I don't know, maybe two weeks almost, that seems like a time where we should have recrawled, at least some of those pages, and shown those again. So that's something where I would say, if we're not indexing anything after two weeks, that feels like maybe there might be something else that is wrong. So maybe something like a removal request is still pending somewhere within the website. You can check that in Search Console. Maybe there is a manual action that's in place. You can also check that in Search Console. Those are, I think, the most obvious ones that could be at role there. It could also be that maybe the website is returning a 404 status code when you look at the page. And in a browser it can look normal, but the status code might be 404. So that's one thing there. But that's really, I would say, the case if we don't index anything at all from the website after those two weeks. If we index things like the home page but not all of the detail pages, then I think that's normal after two weeks. I think that's something that tends to take a little bit longer, especially if the whole website was on noindex for a longer time. So that's something where I'd say like, if we're not indexing anything after two weeks, that feels like something else is wrong. If we're not indexing everything after two weeks, that's completely normal. I would expect that to take maybe several months to get picked up completely again. So those are, I think, the two aspects there. BILAL AHMED: Yeah, all right, I'll take that. Yeah, thank you. JOHN MUELLER: Sure. MIHAI APERGHIS: Hey again. JOHN MUELLER: Hi. MIHAI APERGHIS: So-- oh, happy new year, by the way, John and everyone. So I've got one around Search Console. I've noticed in some cases that certain queries that are very low in average position-- 60, 70, 80-- have a very, very high number of impressions. I know that, for example, with other products like Google Analytics, Google tries to filter out bots and scrapers and things like that. I wonder if for Search Console these high number of impressions for such a low position, where users wouldn't usually get to like page 7 of the search results, are either like-- is it just creepers or bots or anything like that? Or it might be a bug or something. How reliable are these impressions, basically. JOHN MUELLER: We do filtering for kind of scrapers and things like that on several levels. But it's also something that-- it can happen that it shows up in Search Console. And so that's not completely excluded that we would never show that there. But for the most part, I think we-- most of these things are filtered before they reach Search Console. So my guess is, if you're seeing something like that which is obviously kind of a pattern of scrapers and bots and the usual stuff, then that might be kind of a side effect of that, that we're just not filtering in Search Console. But no-- MIHAI APERGHIS: The weird thing is that-- so I'm comparing two queries. But for one of them, the average position is like 10, 11, 12, something like that. The other one, average position is 60 or 70. Both seem to have about the same search volume, give or take. And they both have 10,000, 15,000 expressions. I get why the one on page 1 or page 2 might have that, but page 7, same number of impressions, there has to be something wrong there. JOHN MUELLER: I don't know. It's really hard to say. Sometimes, the ones I've seen that have gotten through in Search Console are things that are more periodic, in that, if you look at the individual query, then you see, oh, it's like every Sunday they come and it's like thousands of impressions, that might be a sign that it's in that direction. But it's really hard to say. Sometimes these things just make it into a Search Console. MIHAI APERGHIS: Right. The only problem with that is that, if you're trying to check out clickthrough rate, for example, to try to figure out, oh, look these are some queries that have a lot of impressions, we have a very low click rate, let's try to optimize [INAUDIBLE] or something like that, where the data is not actually there. It's not actual users that are creating those impressions. JOHN MUELLER: Yeah, I think that's always a struggle. But I mean, on the one hand, I'm not really a fan of people scraping Search anyways. But these bots and scrapers are around all the time anyway, so it's always-- regardless of how you're tracking those metrics, you always have this level of noise that's involved there which makes it a little bit tricky to understand, are these actual users or are these bots acting like users? MIHAI APERGHIS: Right, OK. JOHN MUELLER: Cool. OK, let me pause the recording here. It's been great having you all here. And it's been good doing these Hangouts again, kind of something a little bit regular and normal along the way. I'll be around a little bit longer, as well, so if any of you want to stick around after the recording stops, you're welcome to stick around. Otherwise, I set up the next English Office Hours for next Friday in the evening our time here, so maybe a little bit better for the US time zones. So you don't have to get up in the middle of the night, Michael. Always good to have you here, though. And I think a German one is lined up as well for next week. So if you have anything that we still need to cover, feel free to jump into one of those Office Hours. Or of course, drop by the Search Central Help Forums, where experts like Mihai hang out and can help answer your questions as well. All right, thanks, everyone. And let me pause the recording now.