Important Image & Video File Attachments

[Eric]

Hello. Another site issue is rearing its head and I need to deal with it. The issue is hard drive space and attachment file storage. With the type of hosting I have been using since last November, the hard drive space the site uses is much faster, but space is more limited. That space is being filed at a rate that is starting to be a problem.

Movie files are the biggest file attachments being stored here, by a very large margin. I've just been looking and I was surprised by how large some of these files are and by how much total space they are occupying. I know there are some of you that like to upload movie files here. The option has been turned off since the recent software upgrade, due to some early stability concerns. I am sorry, but that feature will not be coming back. It just uses too much of the site's available resources. It is free and easy to set up an account on YouTube and you can then link to the files here. File sizes aside, the bandwidth to stream those movies also uses a lot of resources. If the videos are hosting by YouTube, it is not my bandwidth being used to serve them. I hope you all understand.

When I first started the site, I did not place file size limits on attachment files. We were on a much more expensive server then and hard drive space was not an issue. Now, there are some individual accounts using significant portions of the sites available storage capacity. One account, in particular, is using 10% of the server's total hard drive space. I was sahocked by this, when I was looking at the sotrage stats this morning.This isn't anyone's fault, but mine. I set the standards and I have allowed this unlimited upload of attachments. I need to start imposing some limits to the size and total number of file attachments allowed, per user and per user group. I won't be making any changes on this right now and when I do, the settings will be generous. Let me give you an example: A 10mb limit on the size of a single picture uploaded is more than fair. It is, in fact, five times as much as many of the other sites I use allow. When looking at the files stored on the server today though, I found a single picture image that was almost ten times that size and I found more than five-hundred that were 30mb or large.

I am going to write a program to go through the file attachment directories that are older than two years, to resize the images stored there. That will buy some breathing room. This isn't a crissis if I start addressing it now. It will start becoming one in as little as a month, if I don't. I just wanted to let you all know what was going on and why video hosting will not be returning to TBS. Thanks.

[MO Fugga]

Dammit, @pipedreams you're hogging all the bandwidth!

[Eric]

OK, the image resizing program is up and running. It won't affect any images posted after 2018 and the older images will simply be resized so that they are no larger than 750 pixels, in either dimension. They will still be viewable. This will take a long time to run and it uses a good bit of server load, but it shouldn't cause any slowdowns on the site.

If any of you see anything weird happening, or have any get any errors, please let me know.

[pipedreams]

Just now, MO Fugga said:

Dammit, @pipedreams you're hogging all the bandwidth!

I hope not, I don't post videos, GIF's  and large files.  One video can equal several hundred pictures.

[Eric]

The test run I was running against a 2gb directory of test images netted a 28% file size savings. That is pretty significant.

[Eric]

Guys, I'll say again that the only person at fault for this situation is me and I can rectify with little or no impact to the site's operation, or your user experience.

I need more tech friends. I think I probably overshare the nuts-and-bolts info on stuff like this because I don't have another outlet for my shop talk. I didn't bring any of this up to poin t a finger at anyone. I was just explaining stuff that most of you don't want or need to hear. Sorry about that.

 

[crockett]

Just found this topic.

The main problem are modern smart phones. My iPhone X takes 12 mbit photos. My next and current phone can take 108 mbit photos. Same with videos recorded with cell phones, many now can create 4k videos. Both creates HUGE files.

It comes at no surprise that both bandwidth and storage space becomes an issue, fast.

As Eric said, most of us have a Google / YouTube account. Uploading videos to YouTube is easy and goes fast. Once that is done, just copy & paste the URL to your YouTube video into your post on the forum.

If you have a cell phone with a large photo sensor, try to take photos with smaller resolution for the forum.

 

[Eric]

Well, that actually went smoothly and I am already done reducing file dimensions and sizes of old images. So that buys some breathing room for me to decide on file upload size limits that won't cramp people's styles. I won't do anything further on this without giving you all a heads-up.

[railfancwb]

Sometimes I quote a post to respond. If that post has picture(s) do they once again use bandwidth and storage?

[Eric]

Just now, railfancwb said:

Sometimes I quote a post to respond. If that post has picture(s) do they once again use bandwidth and storage?

No, just when it is uploaded. The image gets cache on the viewers' systems as well, so we don't take a bandwidth hit for it to be viewed in multiple places. Good question though.