Conversation
supportHello I liked the plugin and I instaled it on my website here ( https://webzy.gr/ ) but the problem is that I get the content only on some of the pages. For all pages, project pages and posts I get the header stuff…. Home page works : https://webzy.gr/index.md Blog page works: https://webzy.gr/webzy-blog.md About page does not work: https://webzy.gr/about-webzy.md None of the post pages work: https://webzy.gr/efarmogi-toy-ai-sto-web-development.md Portfolio doesn’t work: https://webzy.gr/webzy-portfolio/ Portfolio projects don’t work: https://webzy.gr/project/kataskeyi-ilektronikoy-katalogoy-epaggelmatikon-michanon-trofimon.md I turned on debuging which didn’t help. I added Document Root Selectors but that didn’t help. I also removed Ignore Selectors and that also didn’t help. Am I doing something wrong? FYI: I made a change in the code on line 802 file: class-renderer.php to this since I wanted to remove id & published_at and add yoast seo title: private function build_front_matter(WP_Post $post, string $canonical_url, string $markdown_url): array { $data = [ 'title' => html_entity_decode((string) get_post_meta( $post->ID, '_yoast_wpseo_title', true ), ENT_QUOTES | ENT_HTML5, 'UTF-8'), //'title' => html_entity_decode((string) get_the_title($post), ENT_QUOTES | ENT_HTML5, 'UTF-8'), 'description' => html_entity_decode((string) get_post_meta( $post->ID, '_yoast_wpseo_metadesc', true ), ENT_QUOTES | ENT_HTML5, 'UTF-8'), //'id' => (string) $post->ID, 'type' => (string) $post->post_type, 'slug' => (string) $post->post_name, //'published_at' => (string) get_post_time('c', true, $post), 'modified_at' => (string) get_post_modified_time('c', true, $post), 'url' => $canonical_url, 'markdown_url' => $markdown_url, ]; but I don’t think this created some problem. Is there anything I can do to fix the problem? Tank you in advance George The page I need help with: [ log in to see the link]
I’d need to know your settings to help. Have you tried setting your “Document Root Selector” to “html”?
Hello Mr. Michael Thank you for your promt reply on Sunday! I set the document root selector to html also but that didn’t help My settings are as follows: Enabled Post Types: Post (post) , Page (page) , Project (ohio_portfolio) , Structured Data (saswp) , Testimonial (wpm-testimonial) Document Root Selector: main, article, #content, #main-content, #app, html Ignore Selectors: header, footer, nav, form Respect Noindex: Checked Off Include Images: Checked On Could you please send me different senarios so I can try? I disabled any caching if that could be a problem but that didn’t help. I don’t know if the following says something to you but when I enable wordpress debugging I get the following message: Warning : Cannot modify header information – headers already sent by (output started at /usr/www/users/………………../webzy.gr/wp-includes/functions.php:6170) in /usr/www/users/……………./webzy.gr/wp-content/plugins/llm-markdown/includes/class-plugin.php on line 293 Warning : Cannot modify header information – headers already sent by (output started at /usr/www/users/………………../webzy.gr/wp-includes/functions.php:6170) in /usr/www/users/……………./webzy.gr/wp-content/plugins/llm-markdown/includes/class-plugin.php on line 294 Warning : Cannot modify header information – headers already sent by (output started at /usr/www/users/………………../webzy.gr/wp-includes/functions.php:6170) in /usr/www/users/……………./webzy.gr/wp-content/plugins/llm-markdown/includes/class-plugin.php on line 295 Warning : Cannot modify header information – headers already sent by (output started at /usr/www/users/………………../webzy.gr/wp-includes/functions.php:6170) in /usr/www/users/……………./webzy.gr/wp-content/plugins/llm-markdown/includes/class-plugin.php on line 296 Any ideas? Thanks again George
can you set the “Document Root Selector” to just “html” — see if it works then
I just did and that din’t work. When I tried before on some of the pages I got the content like the https://webzy.gr/index.md , https://webzy.gr/about-webzy.md but now not even for those. I put back the main, article, #content, #main-content, #app, html but that didn’t work. Is there any chance that the execution of the rendering script needs server recources which don’t exist at some moments? Is there any chance that the browser, the server and or wordpress caches those pages? I am willing to try different aproaches…. Thanks George
Hiya. So, I just did a little debugging on my local machine. I hardcoded the: $canonical_url = 'https://webzy.gr/'; in function fetch_rendered_html and it works perfectly fine. The plugin is actually pretty simple: it uses wp_safe_remote_get to do a loopback request to the server, pulls the HTML, and then processes the html back into markdown. If you are not getting any markdown, it’s going to be one of these issues: 1 – your server doesn’t support loopback connections. if the webserver webzy.gr can’t call webzy.gr, then it’s just not going to work. 2 – there is some kind of network interference or security layer within your network that is getting in the way of #1. 3 – it might be working, but it’s cached. It uses transients for any non-logged-in-user — so if you are testing in a private window, you may not be seeing the real results. When you are testing, always be logged into wp-admin. Hopefully that helps!
basically, once this works, the plugin will work: $response = wp_safe_remote_get('https://webzy.gr/', [ 'timeout' => 15, 'redirection' => 3, 'sslverify' => true, 'reject_unsafe_urls' => true, 'headers' => [ 'Accept' => 'text/html,application/xhtml+xml', 'X-LLMMD-Render-Source' => '1', ], 'user-agent' => 'LLMMD/1.0.0', ]);
Good morning Once again thank you very much for looking into this. The fact is that I have installed Anti-Spam by CleanTalk and as soon as I disable that plugin I get the content even from a private browser window without beeing logged in. Now it works just fine! Investigating further I enabled the Anti-SPAM by CleanTalk and checked off the Anti-Crawler option. It also works fine! Having the Anti-Crawler option on and in the advanced options of Anti-SPAM by CleanTalk selecting the SpamFireWall feature to off it also works fine! The problem here is that as soon as I disable the SpamFireWall the Anti-Crawler option also gets disabled. SpamFireWall Details: SpamFireWall is a part of the anti-spam service and blocks the most spam active bots before the site pages load. Anti-Crawler is an add-on to SFW and helps to strengthen the protection against spam bots. Disabled by default. Anti-Spam by CleanTalk Anti-Flood is also an add-on to SFW and limits the number of pages visited per minute. Disabled by default. You can read more about SFW modes : https://cleantalk.org/help/anti-flood-and-anti-crawler?utm_source=apbct_hint_sfw__enabled&utm_medium=WordPress&utm_campaign=ABPCT_Settings I tried disabling and enabling different settings on the advanced options when I have the Anti-Crawler option on but nothing worked. I don’t really know which option is the one that stops the page content for rendering. Sorry…. Thank you very much again The support is really excellent! George P.S. : I marked this thread as resolved
Hello @its4yougr I’m with CleanTalk Support. We came across this thread and looked into it on our end. Regarding the blocking issue you experienced, we found only one false positive in our logs related to Anti-Crawler – it blocked the IP 49.13.175.157 . You can add this IP to your SpamFireWall personal lists to prevent future blocks, and enable the Anti-Crawler option after that. Adding a specific IP will not affect regular visitors or search engine indexing. Here is a guide on adding IPs to blocklists: https://cleantalk.org/help/sfw-blacklist-usage You can also check the IP reputation here: https://cleantalk.org/blacklists/49.13.175.157 If you have any questions, we are always happy to answer them through our ticket system: https://cleantalk.org/my/support/open
I’d need to know your settings to help. Have you tried setting your “Document Root Selector” to “html”?
Hello Mr. Michael Thank you for your promt reply on Sunday! I set the document root selector to html also but that didn’t help My settings are as follows: Enabled Post Types: Post (post) , Page (page) , Project (ohio_portfolio) , Structured Data (saswp) , Testimonial (wpm-testimonial) Document Root Selector: main, article, #content, #main-content, #app, html Ignore Selectors: header, footer, nav, form Respect Noindex: Checked Off Include Images: Checked On Could you please send me different senarios so I can try? I disabled any caching if that could be a problem but that didn’t help. I don’t know if the following says something to you but when I enable wordpress debugging I get the following message: Warning : Cannot modify header information – headers already sent by (output started at /usr/www/users/………………../webzy.gr/wp-includes/functions.php:6170) in /usr/www/users/……………./webzy.gr/wp-content/plugins/llm-markdown/includes/class-plugin.php on line 293 Warning : Cannot modify header information – headers already sent by (output started at /usr/www/users/………………../webzy.gr/wp-includes/functions.php:6170) in /usr/www/users/……………./webzy.gr/wp-content/plugins/llm-markdown/includes/class-plugin.php on line 294 Warning : Cannot modify header information – headers already sent by (output started at /usr/www/users/………………../webzy.gr/wp-includes/functions.php:6170) in /usr/www/users/……………./webzy.gr/wp-content/plugins/llm-markdown/includes/class-plugin.php on line 295 Warning : Cannot modify header information – headers already sent by (output started at /usr/www/users/………………../webzy.gr/wp-includes/functions.php:6170) in /usr/www/users/……………./webzy.gr/wp-content/plugins/llm-markdown/includes/class-plugin.php on line 296 Any ideas? Thanks again George
can you set the “Document Root Selector” to just “html” — see if it works then
I just did and that din’t work. When I tried before on some of the pages I got the content like the https://webzy.gr/index.md , https://webzy.gr/about-webzy.md but now not even for those. I put back the main, article, #content, #main-content, #app, html but that didn’t work. Is there any chance that the execution of the rendering script needs server recources which don’t exist at some moments? Is there any chance that the browser, the server and or wordpress caches those pages? I am willing to try different aproaches…. Thanks George
Hiya. So, I just did a little debugging on my local machine. I hardcoded the: $canonical_url = 'https://webzy.gr/'; in function fetch_rendered_html and it works perfectly fine. The plugin is actually pretty simple: it uses wp_safe_remote_get to do a loopback request to the server, pulls the HTML, and then processes the html back into markdown. If you are not getting any markdown, it’s going to be one of these issues: 1 – your server doesn’t support loopback connections. if the webserver webzy.gr can’t call webzy.gr, then it’s just not going to work. 2 – there is some kind of network interference or security layer within your network that is getting in the way of #1. 3 – it might be working, but it’s cached. It uses transients for any non-logged-in-user — so if you are testing in a private window, you may not be seeing the real results. When you are testing, always be logged into wp-admin. Hopefully that helps!
basically, once this works, the plugin will work: $response = wp_safe_remote_get('https://webzy.gr/', [ 'timeout' => 15, 'redirection' => 3, 'sslverify' => true, 'reject_unsafe_urls' => true, 'headers' => [ 'Accept' => 'text/html,application/xhtml+xml', 'X-LLMMD-Render-Source' => '1', ], 'user-agent' => 'LLMMD/1.0.0', ]);
Good morning Once again thank you very much for looking into this. The fact is that I have installed Anti-Spam by CleanTalk and as soon as I disable that plugin I get the content even from a private browser window without beeing logged in. Now it works just fine! Investigating further I enabled the Anti-SPAM by CleanTalk and checked off the Anti-Crawler option. It also works fine! Having the Anti-Crawler option on and in the advanced options of Anti-SPAM by CleanTalk selecting the SpamFireWall feature to off it also works fine! The problem here is that as soon as I disable the SpamFireWall the Anti-Crawler option also gets disabled. SpamFireWall Details: SpamFireWall is a part of the anti-spam service and blocks the most spam active bots before the site pages load. Anti-Crawler is an add-on to SFW and helps to strengthen the protection against spam bots. Disabled by default. Anti-Spam by CleanTalk Anti-Flood is also an add-on to SFW and limits the number of pages visited per minute. Disabled by default. You can read more about SFW modes : https://cleantalk.org/help/anti-flood-and-anti-crawler?utm_source=apbct_hint_sfw__enabled&utm_medium=WordPress&utm_campaign=ABPCT_Settings I tried disabling and enabling different settings on the advanced options when I have the Anti-Crawler option on but nothing worked. I don’t really know which option is the one that stops the page content for rendering. Sorry…. Thank you very much again The support is really excellent! George P.S. : I marked this thread as resolved
Hello @its4yougr I’m with CleanTalk Support. We came across this thread and looked into it on our end. Regarding the blocking issue you experienced, we found only one false positive in our logs related to Anti-Crawler – it blocked the IP 49.13.175.157 . You can add this IP to your SpamFireWall personal lists to prevent future blocks, and enable the Anti-Crawler option after that. Adding a specific IP will not affect regular visitors or search engine indexing. Here is a guide on adding IPs to blocklists: https://cleantalk.org/help/sfw-blacklist-usage You can also check the IP reputation here: https://cleantalk.org/blacklists/49.13.175.157 If you have any questions, we are always happy to answer them through our ticket system: https://cleantalk.org/my/support/open