Are the pictures gone for good?

This section will contain posting guidelines for the forum along with any other information that is relevant to this forum.
User avatar
Alan B
100 GW
100 GW
Posts: 7020
Joined: Sep 11, 2010 7:43 am
Location: San Francisco Bay Area, USA
Contact:

Re: Are the pictures gone for good?

Post by Alan B » Mar 15, 2011 4:44 pm

spinningmagnets wrote:Concerning an EMP pulse or a spate of ultra-sunspots...would storing my separate USB back-up drive inside a DIY "Faraday cage" protect it?

I am only passingly familiar with the terms, I don't know the specific benefits...
probably.

x88x
10 kW
10 kW
Posts: 604
Joined: Oct 11, 2010 4:10 pm
Location: MD, USA

Re: Are the pictures gone for good?

Post by x88x » Mar 15, 2011 4:49 pm

spinningmagnets wrote:Concerning an EMP pulse or a spate of ultra-sunspots...would storing my separate USB back-up drive inside a DIY "Faraday cage" protect it?
It might, but tbh it's gonna be about as much use as that tin-foil hat. ;) Modern HDDs are incredibly robust, and really about the only things you have to worry about 99% of the time are bad firmware (though that's not nearly as prolific a problem as some might make it out to be...I know of one HDD line in the last ~6-7 years that has had any issues..and iirc, the manufacturer replaced any drives with the known problem free of charge), age, and user error. Plus, flash storage isn't magnetic, so as long as it isn't in use at the time, no EMP is ever going to affect it.
That we enjoy great advantages from the inventions of others, we should be glad of an opportunity to serve others by any invention of ours, and this we should do freely and generously.
--Benjamin Franklin
Battery stats comparison spreadsheet.

rebelpilot
1 kW
1 kW
Posts: 337
Joined: Jul 28, 2010 12:44 pm
Location: Ontario, Canada

Re: Are the pictures gone for good?

Post by rebelpilot » Mar 15, 2011 5:59 pm

.
Last edited by rebelpilot on Aug 05, 2011 5:22 pm, edited 1 time in total.

User avatar
auraslip
1.21 GW
1.21 GW
Posts: 3719
Joined: Mar 05, 2010 5:19 pm

Re: Are the pictures gone for good?

Post by auraslip » Mar 15, 2011 6:05 pm

(though that's not nearly as prolific a problem as some might make it out to be...I know of one HDD line in the last ~6-7 years that has had any issues..and iirc, the manufacturer replaced any drives with the known problem free of charge)
Seagate? Yeah....... that was fun ignoring the warnings via newegg reviews to up grade the firmware or risk bricking. One would assume a manufacturer wouldn't ship a hard drive with a known firm ware issue, but thousands of bricked hard drives later....
THANKS TO EVERYONE HERE WHO TAUGHT ME ABOUT EBIKES. I'M IN YOUR DEBT.

I sell Ebike Brake levers and throttles with custom connectors crimped on! $10 shipped!

ES facebook group

r/ebikes ebikes on reddit

User avatar
Alan B
100 GW
100 GW
Posts: 7020
Joined: Sep 11, 2010 7:43 am
Location: San Francisco Bay Area, USA
Contact:

Re: Are the pictures gone for good?

Post by Alan B » Mar 15, 2011 6:28 pm

A good EMP burst will create high voltage pulses in all exposed conductors. Anything susceptible to voltage will pop. Which is pretty much everything made from semiconductors. Doesn't have to be on. Or plugged in. The real question is how much EMP. If it is enough, pop go all the transistors that read out the data. Most equipment is not hardened. It takes lots of extra parts and design to harden against this. All consumer stuff is made for low cost, this kind of issue is ignored.

Hard drives are fairly unreliable, if you have many of them. The recent firmware problems turned out to have affected many years of production of drives. My own personal raid six server has five drives, and they all suffer this firmware problem. They are server grade drives, the best made by a top vendor. The particular bug is not easy to trigger but if triggered the drive goes offline and won't come back. It can be fixed but it is easy to lose the data if not very careful. Just in my own gear, I've had a couple hard drives fail in the Tivo, a couple in laptops (not dropped), and we had a top of the line 10,000 RPM fibrechannel drive pop just a couple weeks ago in one of our servers at work. This is a small fraction of the stuff I see. The price of hard drives and the technology in them has pressed the market very hard and the margins are thin. Seems to me that many years ago when hard drives were a lot more expensive that we had fewer failures. But it is hard to compare.

Of course my personal super duper raid six SOHO server is offline now because a fan is going bad. If it is run long enough it might cook all five drives. It is supposed to protect itself against that kind of problem, but I turned it off till I get another fan to be sure. Of course turning it off raises the possibility that the firmware problem will surface because it has to do with a power cycle when the error log is exactly a certain length.

Anyway, suffice to say, over the last 30 plus years I've seen a lot of folks who lost a lot of data on those reliable hard drives. The new ones may be more reliable, but it doesn't appear so, and when they crash they lose a lot more data than the old ones.

Not far from here, in Oakland, CA a few years ago there was a big fire. The kind of thing you'd expect "cannot happen". Lots of hard drives were melted into slag as people ran from their homes. Many people also didn't make it, and this in a modern US city.

The recent events in Japan undoubtedly also caused an incredible data loss. It is hard to be prepared for everything. If you back all your important data into the cloud, it is not clear that raid is really needed. But for me, there is too much to do it all and so a local raid server is most cost effective compromise. I still put a fair amount in the cloud and am moving more that way as time goes on. One approach to the distribution problem is to get a friend in a remote city to mirror with. Raid six servers mirrored (or backed up) across a distance is pretty good. If a fumble finger doesn't erase the wrong thing, and the delete gets mirrored. And use different brands of disks and perhaps even servers. Still a software flaw in the mirror can get ya. If you have a lot of data, you might not notice the problem for years. Then "where did those old photos go?"

Since this subtle seagate bug had been in their drives for years, and not discovered, it was quite a surprise for everyone. According to experts I know or have read, the full extent of the problem has never been fully acknowledged. But I haven't followed it lately. Fixing the firmware in my drives is not easy, I don't think it can be done in the server they're in, and the process itself runs some risk of bricking the drive.

User avatar
auraslip
1.21 GW
1.21 GW
Posts: 3719
Joined: Mar 05, 2010 5:19 pm

Re: Are the pictures gone for good?

Post by auraslip » Mar 15, 2011 7:55 pm

A question about lost pictures leads to in depth conversation about EMP hardened hard drives and firmware issues.

I want to hug this server so much :D
THANKS TO EVERYONE HERE WHO TAUGHT ME ABOUT EBIKES. I'M IN YOUR DEBT.

I sell Ebike Brake levers and throttles with custom connectors crimped on! $10 shipped!

ES facebook group

r/ebikes ebikes on reddit

alpine44
1 mW
1 mW
Posts: 14
Joined: May 01, 2011 9:06 pm

What is the issue with images not displaying?

Post by alpine44 » May 10, 2011 2:47 pm

In many posts the pictures are not displaying. Is this a browser issue or do the images 'expire' after a while?

User avatar
TylerDurden
100 GW
100 GW
Posts: 7543
Joined: Jan 04, 2007 5:50 pm
Location: Wear the fox hat.
Contact:

Re: What is the issue with images not displaying?

Post by TylerDurden » May 10, 2011 2:51 pm

We had a server crash. Not all the stuff is back yet.
Have a Nice Day,

TD

Image
___________________________________________________________

FYI: Adding pictures?

Bored?... take a crack at the unanswered posts

Please post your Watts-at-speed in the survey.



Image

User avatar
dogman dan
100 GW
100 GW
Posts: 34288
Joined: May 17, 2008 12:53 pm
Location: Las Cruces New Mexico USA

Re: What is the issue with images not displaying?

Post by dogman dan » May 10, 2011 3:37 pm

Hopefully most will eventually be restored. But it will take a lot of peoples expensive "free" time.
Those who made the posts could help, by editing new copies of the pics in. Unfortunately, mine are all in a laptop that died.

Harold in CR
1 MW
1 MW
Posts: 1772
Joined: Feb 01, 2010 7:19 pm
Location: Costa Rica

Re: What is the issue with images not displaying?

Post by Harold in CR » May 11, 2011 1:14 pm

Actually, I'm having the same problem, only, it is with NEW posts and photos. I get the word (image), where the photo should be. I right click and get view image. I click on that and get a server error that Browser can not find Photobucket server ???

This should not have anything to do with the ES server update, correct ??

To go completely off topic for a second, I have suddenly lost the access to BID on Ebay. I sign in, go to an auction, place my bid price in the window, click place bid, and I get "You have signed out" ??? What the hell is going on ??? I have tried contacting Ebay. They don't care. ?????? Anyone have any ideas ????????
Thanks to Justin, the forum is open source and NON-commercialized.

User avatar
TylerDurden
100 GW
100 GW
Posts: 7543
Joined: Jan 04, 2007 5:50 pm
Location: Wear the fox hat.
Contact:

Re: What is the issue with images not displaying?

Post by TylerDurden » May 11, 2011 1:42 pm

Check the timezone settings on your computer.

It may say the correct time on the clock, but if the timezone (GMT-XX) is wrong, cookies get funky.
Have a Nice Day,

TD

Image
___________________________________________________________

FYI: Adding pictures?

Bored?... take a crack at the unanswered posts

Please post your Watts-at-speed in the survey.



Image

User avatar
dogman dan
100 GW
100 GW
Posts: 34288
Joined: May 17, 2008 12:53 pm
Location: Las Cruces New Mexico USA

Re: What is the issue with images not displaying?

Post by dogman dan » May 12, 2011 6:07 am

I had a new page come up without pics yesterday afternoon after adding a comment. I backed up, reloaded the page, and the pix retruned.

Sometimes the pics don't matter. But on the build threads in the pics and vid section, it's the whole point. Wish I'd bothered to download my pics back to my computer when I lost all mine. Not every pic, but at least pics of all my past variations of ebikes.

Harold in CR
1 MW
1 MW
Posts: 1772
Joined: Feb 01, 2010 7:19 pm
Location: Costa Rica

Re: What is the issue with images not displaying?

Post by Harold in CR » May 12, 2011 6:51 am

I am on Central time, and, I keep getting switched to CR websites. That never bothered anything before, but, maybe it is now ??

I did change the time zone to Central, and, those missing photos did show up. Might have been the same thing Dogman described.

ALL photos should be put on the ES server system. Then, outside sources such as Photobucket would not be affected.

Don't know what happened to ES during the last updates, though ???

Thanks for the tip TylerDurdan 8)
Thanks to Justin, the forum is open source and NON-commercialized.

User avatar
GCinDC
10 GW
10 GW
Posts: 4317
Joined: Jan 23, 2009 11:19 am
Location: Washington, DC

Re: Are the pictures gone for good?

Post by GCinDC » May 18, 2011 11:44 am

i wonder if the pc/server/hard drive used to store the backup was an (or connected to an) HP Media Center.

Mine corrupts 30% of my images. Never found an answer, only others, with similar issues...

The only solution was to never connect a hard drive with images...

Are there no more backups?
Youtube channel, 2013 Highlights vid. Ebike Nerdcast.
Giant DH Comp: 20s lipo (11.6Ah), hs3540, Adaptto Mini-E, 900W BMSBattery charger.
GT I-Drive (sold), 20s lipo, 9C, 72V 45A 12FET.

User avatar
Thud
100 MW
100 MW
Posts: 2809
Joined: Aug 20, 2009 6:20 am
Location: West Michigan,USA

Re: Are the pictures gone for good?

Post by Thud » May 18, 2011 1:11 pm

Being barely computer literate....
I assumed it was poor form to upload all the pictures onto the forum sites.(what do I know from bandwidth???) So I have all my photo's hosted & linked from Photo-bucket & back ups of the photos on a couple flash drives in no particular order LOL.

I lost absolutly nothing from my threads or contribution photo's during the recent glitching. I do beleive all the zip files in the "infineon" programing threads are corupted also. I tried to download onto another lap top last night with no luck.

Any one with the files who could re-post them? it would be great for the comunity.
T
get some......

All information & advice provided by Thud are "Open Source" & free for personal use & distribution under the following agreement linked below.
http://creativecommons.org/licenses/by-nc-nd/3.0/

User avatar
Kingfish
10 GW
10 GW
Posts: 4275
Joined: Feb 03, 2010 11:23 am
Location: Redmond, WA-USA, Earth, Sol, Orion–Cygnus Arm, Milky Way. Age: > yesterday < tomorrow
Contact:

Re: Are the pictures gone for good?

Post by Kingfish » May 19, 2011 10:57 pm

The links to all of my contributions have been retargeted to host from my server.

However I will not re-edit all my posts to fix the corrupt characters: I have already provided a SQL solution to the Admins shortly after the problem became apparent; near as I can tell it has fallen on deaf ears. Cleaning up content is part and parcel to good web development; it’s not rocket science, I gave them the solution, and they haven’t used it. It’s a giant black hole mystery and it’s frustrating :?

Caveat emptor, KF
* My 2WD Garden Wall
* Kinaye MotorSports
* Primary ride: 2WD Disc 9C 2806-equiv / Dual Lyen 12FET / 20S7P LiPo.
* Epics: Going to California: 2011 8)
* 50-mph, 101, 10k-Club. 12,527 miles-to-date, 7037 as 2WD.

It is by caffeine alone I set my mind in motion.
It is by the beans of Java that thoughts acquire speed.
The hands acquire shakes, the shakes become a warning.
It is by caffeine alone I set my mind in motion.

User avatar
amberwolf
100 GW
100 GW
Posts: 23903
Joined: Aug 17, 2009 6:43 am
Location: Phoenix, AZ, USA, Earth, Sol, Local Bubble, Orion Arm, Milky Way, Local Group
Contact:

Re: Are the pictures gone for good?

Post by amberwolf » May 20, 2011 2:00 am

AFAIK the only one with the power to do the fixes is KnightMB, who appears to be too busy with the rest of his life/job/etc. to do anything yet. :(

User avatar
neptronix
100 GW
100 GW
Posts: 13061
Joined: Jun 15, 2010 5:56 pm
Location: People's republic of Petrolistan ( Utah, USA )
Contact:

Re: Are the pictures gone for good?

Post by neptronix » May 20, 2011 2:22 am

Kingfish wrote:The links to all of my contributions have been retargeted to host from my server.

However I will not re-edit all my posts to fix the corrupt characters: I have already provided a SQL solution to the Admins shortly after the problem became apparent; near as I can tell it has fallen on deaf ears. Cleaning up content is part and parcel to good web development; it’s not rocket science, I gave them the solution, and they haven’t used it. It’s a giant black hole mystery and it’s frustrating :?

Caveat emptor, KF
Yeah....... I hear you there. We have a great community here but it seems that the original enthusiasm for this forum is gone.

As for the HD, i'd seen stats from the old server in some other thread and it looked like the old server was an old Athlon ( Athlon XP? ) box which was running way over it's memory limit and swapping like crazy. If the hard drive was circa the processor, i see how corruption could have occurred. Hard drives are good for 5 years, anything after that is a bonus.

The main web site( endless-sphere.com )'s joomla installation is broken too.

The jpeg files are either entirely missing, or missing large contiguous chunks out of their data, which points to bad sectors on the hard drive of the previous server. I think they are goners. I wish we'd get an official word though.
My first major build: 8T MAC motor on a Trek 4500.
The new all-arounder: Leafmotor 1500w on a Turner O2 full suspension.
The wheelie machine: 20" Rear Magic Pie II on a Trek 4300 MTB

"The best time to plant a tree was 20 years ago. The second best time is now."- Chinese Proverb

User avatar
Kingfish
10 GW
10 GW
Posts: 4275
Joined: Feb 03, 2010 11:23 am
Location: Redmond, WA-USA, Earth, Sol, Orion–Cygnus Arm, Milky Way. Age: > yesterday < tomorrow
Contact:

Re: Are the pictures gone for good?

Post by Kingfish » May 20, 2011 9:02 am

I don’t get it: If you are acting as a host, an ISP of some sorts – you must use RAID or some other topology that distributes the load and/or supports failover. A single-drive solution is pissing in the wind and reckless. Sorry: I call it the way I see it, and obviously is now history.

Don’t get me wrong: I understand completely the frustration of hosting alone or finding someone that can be trusted with “your baby”; been there/done that on several occasions. Post-partum is not fun unless there is a reasonable plan to share the workload… or handoff, least it be taken from you.

Mr. KnightMB: You have your ears on buddy? I’m trying to do you a favor. :)

The best bet is to migrate to a dedicated farm, raise the cash from the elite members to offset the cost, and have TWO or more admins capable of providing a modicum of support so that we are not sole-sourced and constrained by the one element.

Cost breakdown for a typical Forum:
  • Web Hosting: $120/year (+/- whatever)
  • SQL Server: $120/year (MySQL might be cheaper… )
  • Domain Registration: Varies $15-35 depending on who is doing it.
    Max Total: $275/yr
  • SSL? Great way to market ES-centric product to raise cash.
What’d I miss?

BTW – I do not seek the Admin role; I’ve plenty on my plate. :) I just want to contribute and converse with kindred folks, and pray my content doesn’t disappear – or worse: Be absconded for private gain.

Please please please consider a way to hand off or share so that we can fix the issues.
Most sincerely, KF
* My 2WD Garden Wall
* Kinaye MotorSports
* Primary ride: 2WD Disc 9C 2806-equiv / Dual Lyen 12FET / 20S7P LiPo.
* Epics: Going to California: 2011 8)
* 50-mph, 101, 10k-Club. 12,527 miles-to-date, 7037 as 2WD.

It is by caffeine alone I set my mind in motion.
It is by the beans of Java that thoughts acquire speed.
The hands acquire shakes, the shakes become a warning.
It is by caffeine alone I set my mind in motion.

bmxer
1 µW
1 µW
Posts: 2
Joined: Jan 30, 2014 10:29 pm

Possible fix for corrupt images?

Post by bmxer » Jan 31, 2014 1:59 pm

Hi all,
Questions about the corrupt images:

Has anyone looked into restoring (script mining) the broken images from older intact copies on archive.org?

Is the exact date known when the images were first corrupted? Was there just a single (known) event that caused it?

Why does it only affect some older images, apparently not all of them?
-------
Haro BMX with 9C rear hub,
LYEN 12 FET controller,
12S 20AH Lipo, 4500 Miles

User avatar
amberwolf
100 GW
100 GW
Posts: 23903
Joined: Aug 17, 2009 6:43 am
Location: Phoenix, AZ, USA, Earth, Sol, Local Bubble, Orion Arm, Milky Way, Local Group
Contact:

Re: Possible fix for corrupt images?

Post by amberwolf » Jan 31, 2014 2:17 pm

There's a thread about that and what we've tried to do about it, including some people that have copies of some of the older images. Unfortunately I don't have a link to it, but I think it's in the forum rules and features subforum.


EDIT: I found a few threads, and am putting them in this subforum (merging whereever appropriate).

One is here:
https://endless-sphere.com/forums/vi ... 13&t=31437
that includes discussion of this problem.

There's also some moderator-forum discussions that I merged together, but not sure if they should also be copied over to this thread. I added a note to that thread for the mods to discuss and decide what to put here.

bmxer
1 µW
1 µW
Posts: 2
Joined: Jan 30, 2014 10:29 pm

Re: Are the pictures gone for good?

Post by bmxer » Jan 31, 2014 10:35 pm

Thanks amberwolf.

It sounds like the database was migrated to another server, probably with different compilation or config options. There is a MySQL config option called backslash escaping which could do this. Different quote escaping in the prior server's SQL dump could cause a bad hair day too.

The original cause is likely one of those issues or a related ome, and is fixable, provided a dump of the original pre-August 31 migration still exists. Does such a backup still exist? If it does I would like to download a copy to look at. It would be nice to see all those older images resurrected. I can extract raw db files, SQL dumps, or whatever form the original backup is in.

Looks like mrvass has not logged in for awhile. Who would know how to find the original Aug 2011 snapshot he created? Is it hidden away in some folder on the new server?
-------
Haro BMX with 9C rear hub,
LYEN 12 FET controller,
12S 20AH Lipo, 4500 Miles

User avatar
Kingfish
10 GW
10 GW
Posts: 4275
Joined: Feb 03, 2010 11:23 am
Location: Redmond, WA-USA, Earth, Sol, Orion–Cygnus Arm, Milky Way. Age: > yesterday < tomorrow
Contact:

Re: Are the pictures gone for good?

Post by Kingfish » Feb 01, 2014 12:44 am

^ All of this has been hashed out before. We've recovered what could be. This horse has been flogged to death and the bones have been picked over. There's no going back. The Site is what it is; accept it for that.

Lastly, I have to believe the new owners have a back up policy that is superior to the previous do-nothing attitude.

Grateful we have anything left over. KF
* My 2WD Garden Wall
* Kinaye MotorSports
* Primary ride: 2WD Disc 9C 2806-equiv / Dual Lyen 12FET / 20S7P LiPo.
* Epics: Going to California: 2011 8)
* 50-mph, 101, 10k-Club. 12,527 miles-to-date, 7037 as 2WD.

It is by caffeine alone I set my mind in motion.
It is by the beans of Java that thoughts acquire speed.
The hands acquire shakes, the shakes become a warning.
It is by caffeine alone I set my mind in motion.

Post Reply