Is it there a way to selectively upload only new items?
-
July 30, 2020 at 11:39 pm #3314[anonymous]
In the first days, when I started using Publii, I found it too fast to publish my page.
With a few weeks using it, I have found the upload takes more and more time, as it publishes the whole website and not only the new or updated/modified posts and its contents.
Is it there a way to just upload the affected new or updated/modified items?
I know that if I publish a new post, the main page and other elements will be modified or affected by this new post. So it would be nice to have these elements updated at the time I upload a new post.
But the rest of the static pages/elements will be the same. So I guess there should be a way to do what I say, maybe already solved but I haven’t noticed that options so far.
Thank you very much.
July 31, 2020 at 7:03 am #3316BobHi geppettvs,
There are many elements that affect the length of the upload process, for example, Related posts – when you add a new post the list of related posts changes on all post pages.
Have a look at the “How to speed up rendering process?” article, it may help you optimize your upload time.
July 31, 2020 at 4:57 pm #3317[anonymous]Hey Bob, thank you very much for your answer.
I understand there will be many things which will be affected by new posts. And I understand that will take a lot of time which I don’t care to spend if that’s a major change to the blog.
But look.
Sometimes I misspell a word, or let’s say: If I want to change a picture, removing an old picture and placing a new one.
If I modify a single character to the newest post, having in mind that everything else will be the same. The affected post has just a character difference, but the whole page will be uploaded again.
I guess the only way to do such thing is publishing my blog to a local folder, then manually uploading via FTP with the proper FTP client, which will ask what to do with current files on the server, so I can answer “If size differs” or not to overwrite current files on the server.
Anyway, as Publii solves by itself the FTP task, it would be great to have this options when uploading a correction like the one I mentioned.
Thank you!
August 1, 2020 at 7:40 am #3323[anonymous]@geppettvs – if Publii uploads many files due small change on the post there are two possibilities:
- Your small change affects many subpages and they should be updated anyway.
- There is a problem on your server with uploading files.publii.json file – this file is used to compare file contents checksums, if file not exists on the list or checksum is different – Publii uploads a file.
August 1, 2020 at 6:21 pm #3325[anonymous]Hello Tomasz.
Thank you for your answer.
Your point 2 sounds technically logic. Nevertheless I haven’t done anything to modify the behavior of Publii on this point.
So now I wonder if there’s something I should have done before in order to make Publii work faster.
If so, can you please point me to an online doc where to find such information?
Let me say that I am just working with the default Publii options. So once I finish writing my new post, or comiting my minor changes to an old post, I just press the “Sync your website” button and when I see the “Website synchronization” window I hit the “Sync your website” again.
I have noticed there is a warning which reads:
Remember! Any duplicate files or filenames already in the destination location that match the Publii-generated files will be overwritten.
Which makes me think that, if -as mentioned before- I change a single character (misspelled word or something), the whole images and other stuff will be uploaded.
This should be solved by the comparison method you mentioned in your point 2, but I don’t know how to check if it is working on my end.
I am using Linux, Ubuntu 18.04 LTS and Publii is running pretty fine on this OS.
Any ideas are welcome.
Thanks in advance.
August 4, 2020 at 7:20 am #3338[anonymous]How many files is uploaded during sync? And how many files has your website at all?
August 4, 2020 at 5:18 pm #3345[anonymous]Hello Tomasz.
I am attaching a screenshot of my Publii uploading the blog. I didn’t commit any change, didn’t change anything. Just opened the application and hit the “Sync your website” button as I originally mentioned.
As you can see, Publii is uploading the whole website. This time it is showing “Uploading website (nnn of 479 operations done)” but If I publish new posts the number of operations will increase.
I mean: the whole website is being uploaded, no matter what I do.
Thank you very much.
August 4, 2020 at 5:33 pm #3348[anonymous][anonymous] wrote:Hello Tomasz.
I am attaching a screenshot of my Publii uploading the blog. I didn’t commit any change, didn’t change anything. Just opened the application and hit the “Sync your website” button as I originally mentioned.
As you can see, Publii is uploading the whole website. This time it is showing “Uploading website (nnn of 479 operations done)” but If I publish new posts the number of operations will increase.
I mean: the whole website is being uploaded, no matter what I do.
Thank you very much.
I am now attaching a new screenshot. Publii finished uploading the whole website 15 minutes after I posted my previous answer.
So, I guess it is not doing the file checksum.
Any ideas?
August 4, 2020 at 6:12 pm #3353[anonymous]Could you check if there is files.publii.json file on your server and if it is not corrupted?
August 5, 2020 at 2:49 am #3359[anonymous]Tomasz,
I have checked on my web server and I can see the file files.publii.json does exist in the root of the blog folder, as you can see highlighted on this screenshot.
The file contents seems not to be corrupted, as I can open this file in nano and the file contents looks human readable, I have checked the rest of the document and everything seems to be fine. Please check this screenshot.
The only thing I can’t understand, is why the elements declarations shows the path as an absolute path and not a relative one.
I mean, it reads:
{ "path": "/1er-sincronizada-2020/index.html", "type": "file", "md5": "0f1c47053ce17ef69adf5ee04b63bb76" },
Where the path is /1er-sincronizada-2020/index.html, nevertheless, the absolute path on the server would be something starting with /home/USERNAME/public_html/blog/…
I am aware that the local paths will be different, based on my preferences and I don’t know if the path is just a part of the full path and the program itself will do something to gather the proper server’s folders information in order to run file checksums.
If this is not the normal behavior and you think I need to do something to fix this problem with the paths, do you mind telling me where can I set this variable?
Thanks in advance.
August 16, 2020 at 6:25 pm #3546[anonymous]Sorry for the late reply – could you check the files.publii.json file with https://jsonlint.com/ to make sure that this file is correct JSON file?
Also – are you using FTP deployment method? Or other one?
August 16, 2020 at 6:44 pm #3549[anonymous]Hey Tomasz, good to read you. I hope you are fine and with good health.
I have validated the files.publii.json file using the link you provided an it resulted in a “Valid JSON” message as you can see in this screenshot.
For what is about the deployment method: yes. I am using the FTP protocol with SSL/TLS in the Server Settings as shown in the Server Settings page, you can see my configuration on the following screenshots. I have changed some sensible data just for the screenshot to avoid problems.
I am not using the “Use relative URLs” because of the warning shown on the foot of this option, which reads: Please remember: while using relative URLs, some features like Open Graph tags, sharing buttons etc. cannot work properly.
If you think this option may solve something just let me know. Please.
Thanks in advance.
August 16, 2020 at 7:57 pm #3554[anonymous]Now I see that you are using older version of Publii (the relative URLs option is temporary hidden and will be back in v.0.37.0) – could you try to use the newest Publii version (0.36.1)?
August 18, 2020 at 1:20 am #3559[anonymous]I have upgraded Publii to Version 0.36.1 as you suggested. The upload process -nevertheless- seems to have only cosmetic changes, that’s all I have noticed so far.
The upload process started at 19:57 and finished at 20:16 and uploaded every single file on the local folder.
So I guess there’s something wrong. Maybe something on my end?
Thank you for your help and patience.
Screenshots attached.
Regards
August 19, 2020 at 8:59 am #3567[anonymous]@geppettvs – could you share in a private reply file deployment-process.log from Tools -> Log viewer?
I would like to check if the app recognizes correctly the publii.files.json
August 19, 2020 at 2:38 pm #3568[anonymous]This reply has been marked as private.August 19, 2020 at 5:00 pm #3571[anonymous]@geppettvs – ok, I see the reason of your issue:
> Malformed files-remote.json file: SyntaxError: Unexpected end of JSON input
So the app downloads your files.publii.json file, but for some reason it is invalid (it stores it under different name, but it is ok).
Could you check the content of the input/config/files-remote.json in your site directory? Maybe during download the file is corrupted in some way.
August 19, 2020 at 6:20 pm #3572[anonymous]There’s no <span style=”color: #ff0000;”>input</span> folder in the server’s blog folder.
A screenshot is attached.
How can I fix this?
I didn’t delete anything, it seems my previous Publii version didn’t create it, and the folders and file does not exist.
August 19, 2020 at 6:29 pm #3574[anonymous]Sorry – I mean input/config/files-remote.json on your local computer in the site directory 🙂
August 19, 2020 at 8:37 pm #3575[anonymous]I am attaching a ZIP file with the files-remote.json file inside.
I can see this file is -somehow- incomplete, truncated at line 1943.
I don’t know how to fix it.
Would deleting/renaming this folder make Publii to re-create the file, thus fixing the issue?
Thank you.
August 20, 2020 at 6:17 pm #3588[anonymous][anonymous] wrote:Hey Tomasz, good to read you. I hope you are fine and with good health.
I have validated the files.publii.json file using the link you provided an it resulted in a “Valid JSON” message as you can see in this screenshot.
For what is about the deployment method: yes. I am using the FTP protocol with SSL/TLS in the Server Settings as shown in the Server Settings page, you can see my configuration on the following screenshots. I have changed some sensible data just for the screenshot to avoid problems.
I am not using the “Use relative URLs” because of the warning shown on the foot of this option, which reads: Please remember: while using relative URLs, some features like Open Graph tags, sharing buttons etc. cannot work properly.
If you think this option may solve something just let me know. Please.
Thanks in advance.
Replace public_html/blog with /public_html/blog/ and you should have no more problems. I had the same problem with one of my domains, after adding a slash at the end, it only uploaded the changed files.
August 22, 2020 at 7:45 am #3596[anonymous]@kosi77 – it is an interesting tip 🙂 I am not sure if it will help but maybe it is worth to try.
I have analysed the logs and it seems that on the FTP level the file is corrupted – Publii downloads 57kb instead of 67kb what causes corrupted JSON file.
And at this moment I have no idea why :/ Because it is a simple download operation using FTP library.
August 22, 2020 at 4:13 pm #3597[anonymous]The inclined line before and after works very well for all my domains.
Without a slash at the end, he uploaded all sorts of files every time.
I don’t know what this is due to, but it works perfectly for me.
August 23, 2020 at 5:00 pm #3604[anonymous]Hello Tomasz, kosi77
I have added the slash at the end of the public_html/blog/ and now the process takes less time, yes.
I initiated the blog sync at 11:44:02 and it finished at 11:45:30, which is much less time than before.
Now the process shows it will do up to 48 operations which is much less than the 479 it used to do before. And these 48 operations takes less than 2 minutes.
I have checked the upload process two times and the results and time to upload are almost the same. But the number of operations remains in 48, which I guess may increase based on the new contents or changes I commit to my blog.
The files-remote.json now seems not to be broken. It looks fine. So I guess the slash at the end of the physical location is a good solution this time.
The tip below the field looks fine but it does not warn about leaving the field without the slash. So I guess I didn’t follow the proper instructions when setting my server up. Maybe a warning on the screen may help?
Thank you very much.
August 23, 2020 at 5:07 pm #3610[anonymous]This reply has been marked as private.August 27, 2020 at 5:29 pm #3638[anonymous]I have created new contents today, in the same conditions I used to have this problem and everything seems to be running fast and smooth this time.
You can see my blog at https://nuvogue-electronics.com.mx/blog/
So we can say the solution was to add the slash at the end of the public_html/blog/ as kosi77 suggested.
Thank you very much. This topic has been solved with your help.
Appreciation.
Regards.