Forum software upgrade on 11/17/2017 technical issues thread

Talk about anything and everything here within reason.
User avatar
neptronix
100 GW
100 GW
Posts: 13063
Joined: Jun 15, 2010 5:56 pm
Location: People's republic of Petrolistan ( Utah, USA )
Contact:

Re: Forum software upgrade happening on 11/16/17

Post by neptronix » Nov 20, 2017 11:25 pm

I've spent some time trying to figure this out today. Not getting anywhere just yet.
Looks like i need to reverse engineer the forum code to see what is going on exactly.
Good times.
My first major build: 8T MAC motor on a Trek 4500.
The new all-arounder: Leafmotor 1500w on a Turner O2 full suspension.
The wheelie machine: 20" Rear Magic Pie II on a Trek 4300 MTB

"The best time to plant a tree was 20 years ago. The second best time is now."- Chinese Proverb

User avatar
neptronix
100 GW
100 GW
Posts: 13063
Joined: Jun 15, 2010 5:56 pm
Location: People's republic of Petrolistan ( Utah, USA )
Contact:

Re: Forum software upgrade happening on 11/16/17

Post by neptronix » Nov 21, 2017 6:41 am

Fixed!

We have been having system instability that seems to be related to our database since this transition. According to the logs, the 20th was really bad. I have been chasing the cause of this for about 2 days now, and will continue on the problem's trail until it is resolved and this site has consistently snappy performance. I believe i might have made a dent in it tonight.. :idea:

Hang in there, and thanks for dealing with the speed bumps thus far.
My first major build: 8T MAC motor on a Trek 4500.
The new all-arounder: Leafmotor 1500w on a Turner O2 full suspension.
The wheelie machine: 20" Rear Magic Pie II on a Trek 4300 MTB

"The best time to plant a tree was 20 years ago. The second best time is now."- Chinese Proverb

User avatar
Alan B
100 GW
100 GW
Posts: 7025
Joined: Sep 11, 2010 7:43 am
Location: San Francisco Bay Area, USA
Contact:

Re: Forum software upgrade happening on 11/16/17

Post by Alan B » Nov 21, 2017 9:09 am

Thanks for all your effort.

Forum asked me to login again, and now I see the mod forum.

User avatar
eCue
100 W
100 W
Posts: 121
Joined: Nov 03, 2017 5:14 pm
Location: Vancouver Island , Canada

Re: Forum software upgrade happening on 11/16/17

Post by eCue » Nov 21, 2017 11:24 am

its nice having the notifications fixed it had never worked for me now it does and he font size fix is working excellent.
"98 Quintana roo Panamint 350w mxus build link - https://endless-sphere.com/forums/view ... =6&t=91627

Addy
10 W
10 W
Posts: 65
Joined: Aug 11, 2017 11:15 am

Re: Forum software upgrade happening on 11/16/17

Post by Addy » Nov 21, 2017 4:18 pm

I'm seeing "[The extension zip has been deactivated and can no longer be displayed.]" in several places. Is this intentional? There are now old posts with missing files because zip files are not allowed.

User avatar
neptronix
100 GW
100 GW
Posts: 13063
Joined: Jun 15, 2010 5:56 pm
Location: People's republic of Petrolistan ( Utah, USA )
Contact:

Re: Forum software upgrade happening on 11/16/17

Post by neptronix » Nov 21, 2017 4:55 pm

eCue wrote:
Nov 21, 2017 11:24 am
its nice having the notifications fixed it had never worked for me now it does and he font size fix is working excellent.
Awesome!
Addy wrote:
Nov 21, 2017 4:18 pm
I'm seeing "[The extension zip has been deactivated and can no longer be displayed.]" in several places. Is this intentional? There are now old posts with missing files because zip files are not allowed.
I have changed the extension settings on the backend. Let me know if this works or not. If it doesn't work, please point me to a thread with a .zip attachment so i can use that for testing.
My first major build: 8T MAC motor on a Trek 4500.
The new all-arounder: Leafmotor 1500w on a Turner O2 full suspension.
The wheelie machine: 20" Rear Magic Pie II on a Trek 4300 MTB

"The best time to plant a tree was 20 years ago. The second best time is now."- Chinese Proverb

Addy
10 W
10 W
Posts: 65
Joined: Aug 11, 2017 11:15 am

Re: Forum software upgrade happening on 11/16/17

Post by Addy » Nov 21, 2017 5:02 pm

neptronix wrote:
Nov 21, 2017 4:55 pm
I have changed the extension settings on the backend. Let me know if this works or not. If it doesn't work, please point me to a thread with a .zip attachment so i can use that for testing.
Thanks, it seems to be working again. Here is a thread with some zip files for example:
https://endless-sphere.com/forums/viewt ... =3&t=90058

Hillhater
100 GW
100 GW
Posts: 8332
Joined: Aug 03, 2010 10:33 pm
Location: Sydney ..(Hilly part !) .. Australia/ Down under !

Re: Forum software upgrade happening on 11/16/17

Post by Hillhater » Nov 21, 2017 5:33 pm

Locked out again trying to post.
"Unable to access network" is the error message.
All other sites etc function normally.
Shutdown and reload ES site needed to restore access ?

We seem to have 2 threads running for these issues,..can they be combined .
This forum owes its existence to Justin of ebikes.ca

User avatar
neptronix
100 GW
100 GW
Posts: 13063
Joined: Jun 15, 2010 5:56 pm
Location: People's republic of Petrolistan ( Utah, USA )
Contact:

Re: Forum software upgrade happening on 11/16/17

Post by neptronix » Nov 21, 2017 7:11 pm

Performance issues are still ongoing. I still only have limited clues as to the cause.
Locked down the DDOS protection a bit further, upgraded the database server, and will continue to watch the system for the rest of the day.
My first major build: 8T MAC motor on a Trek 4500.
The new all-arounder: Leafmotor 1500w on a Turner O2 full suspension.
The wheelie machine: 20" Rear Magic Pie II on a Trek 4300 MTB

"The best time to plant a tree was 20 years ago. The second best time is now."- Chinese Proverb

User avatar
neptronix
100 GW
100 GW
Posts: 13063
Joined: Jun 15, 2010 5:56 pm
Location: People's republic of Petrolistan ( Utah, USA )
Contact:

Re: Forum software upgrade happening on 11/16/17

Post by neptronix » Nov 21, 2017 10:18 pm

Hillhater wrote:
Nov 21, 2017 5:33 pm
Locked out again trying to post.
"Unable to access network" is the error message.
All other sites etc function normally.
Shutdown and reload ES site needed to restore access ?

We seem to have 2 threads running for these issues,..can they be combined.
I have renamed them so that their purpose is a little more distinct.

Yes, you saw the instability i was talking about.
Still working on it. I have a couple leads on the cause. The problem is that it's intermittent and rare.. the worst kind of issue.

Peak load on the server happens in 2 hours from now though, so we should know by then if the changes i made today resolved the issue or not. Join me in the crossing of fingers. :lol:
My first major build: 8T MAC motor on a Trek 4500.
The new all-arounder: Leafmotor 1500w on a Turner O2 full suspension.
The wheelie machine: 20" Rear Magic Pie II on a Trek 4300 MTB

"The best time to plant a tree was 20 years ago. The second best time is now."- Chinese Proverb

User avatar
neptronix
100 GW
100 GW
Posts: 13063
Joined: Jun 15, 2010 5:56 pm
Location: People's republic of Petrolistan ( Utah, USA )
Contact:

Re: Forum software upgrade on 11/17/2017 technical issues thread

Post by neptronix » Nov 22, 2017 12:30 am

It's 10am and this is usually the witching hour for our server. This is peak load. I have seen the system go tits up twice during this hour.
Not a hiccup and things seem super snappy.

I like what i see 8)

Let me know if you experience any slow or funky response from now on.. fingers crossed that i nailed the stability issue to the wall :mrgreen:
My first major build: 8T MAC motor on a Trek 4500.
The new all-arounder: Leafmotor 1500w on a Turner O2 full suspension.
The wheelie machine: 20" Rear Magic Pie II on a Trek 4300 MTB

"The best time to plant a tree was 20 years ago. The second best time is now."- Chinese Proverb

User avatar
fechter
100 GW
100 GW
Posts: 11822
Joined: Dec 31, 2006 3:23 pm
Location: California Bay Area, USA

Re: Forum software upgrade on 11/17/2017 technical issues thread

Post by fechter » Nov 22, 2017 9:35 am

As of right now, I'm seeing response times of 10-15 sec. Pretty slow.

Unrelated: is there any possibility of having a "Quick links" at the bottom of the page? Always have to scroll up to find it.
"One test is worth a thousand opinions"

User avatar
Alan B
100 GW
100 GW
Posts: 7025
Joined: Sep 11, 2010 7:43 am
Location: San Francisco Bay Area, USA
Contact:

Re: Forum software upgrade on 11/17/2017 technical issues thread

Post by Alan B » Nov 22, 2017 10:06 am

Yesss on response and request for Quick Links the the bottom!!

User avatar
teklektik
1.21 GW
1.21 GW
Posts: 3524
Joined: Mar 26, 2011 1:15 pm
Location: CT, USA

Re: Forum software upgrade happening on 11/16/17

Post by teklektik » Nov 22, 2017 11:06 am

Hillhater wrote:
Nov 21, 2017 5:33 pm
Locked out again trying to post.
We seem to have 2 threads running for these issues,..can they be combined .
neptronix wrote:
Nov 21, 2017 10:18 pm
I have renamed them so that their purpose is a little more distinct.
Might have been useful to combine the slow response thread since it's not getting any traction, but it was helpful getting this thread renamed to address technical issues instead of just appearance and feature tweaks.

That said, response is really slow today - 10 seconds to bring up a thread and much worse to get a post preview or submit a post. These are very atypical response times compared to other inet links so this is not a provider or local equipment issue.
Visit Grin Technologies at www.ebikes.ca
Build Thread: 2WD Yuba Mundo V4

User avatar
neptronix
100 GW
100 GW
Posts: 13063
Joined: Jun 15, 2010 5:56 pm
Location: People's republic of Petrolistan ( Utah, USA )
Contact:

Re: Forum software upgrade on 11/17/2017 technical issues thread

Post by neptronix » Nov 22, 2017 4:03 pm

Hi folks.. just woke up a bit ago and spent a few hours trying to figure out what the performance issue is now.

The good news is that our system has managed to remain consistently online, which is a first. Previously, our database was taking a dump due to being out of memory, and periodically causing the site to completely lock up. That issue seems to be cured.

The bad news is that this new huge database still cannot handle what's being thrown at it, hence the slowness.
I am beginning to wonder if this new phpbb is a mega database performance hog... or what.

Performance will be poky today. Late in the night, i will be doing surgery on the database.
Thank you for remaining patient with me. We are getting closer to a resolution for this.
My first major build: 8T MAC motor on a Trek 4500.
The new all-arounder: Leafmotor 1500w on a Turner O2 full suspension.
The wheelie machine: 20" Rear Magic Pie II on a Trek 4300 MTB

"The best time to plant a tree was 20 years ago. The second best time is now."- Chinese Proverb

User avatar
DrkAngel
10 GW
10 GW
Posts: 4468
Joined: Dec 15, 2010 11:14 am
Location: Upstate-Western-Southern Tier NY. USA

Re: Forum software upgrade on 11/17/2017 technical issues thread

Post by DrkAngel » Nov 22, 2017 10:03 pm

Is the ES wiki on the schedule?
Disappeared when ES came back on line.

https://endless-sphere.com/w/index.php/Main_Page
A little learning is a dangerous thing;
Drink deep, or taste not the Pierian spring:
There, shallow draughts intoxicate the brain,
And drinking largely, sobers us again.

I enjoy enlightening ... and enlivening the spirit of the innovators.

New & Improved - Acronym Definitions

Index - Homemade Battery Packs - Updated - November 2015

EBike Toolbox - Bargains! $

Endless Sphere Wiki

User avatar
neptronix
100 GW
100 GW
Posts: 13063
Joined: Jun 15, 2010 5:56 pm
Location: People's republic of Petrolistan ( Utah, USA )
Contact:

Re: Forum software upgrade on 11/17/2017 technical issues thread

Post by neptronix » Nov 22, 2017 11:14 pm

DrkAngel wrote:
Nov 22, 2017 10:03 pm
Is the ES wiki on the schedule?
Disappeared when ES came back on line.

https://endless-sphere.com/w/index.php/Main_Page
From what i can tell, I had the wiki working the next day after the transition occurred.. are you having troubles with it? clicking the link you provided brings it up. The only issue i know of is that there is a missing logo in the top left.

Not many people use the wiki, so if there is something wrong with it i'm not noticing, let me know. I enver really used it, so i don't know how it should be operating.
My first major build: 8T MAC motor on a Trek 4500.
The new all-arounder: Leafmotor 1500w on a Turner O2 full suspension.
The wheelie machine: 20" Rear Magic Pie II on a Trek 4300 MTB

"The best time to plant a tree was 20 years ago. The second best time is now."- Chinese Proverb

User avatar
DrkAngel
10 GW
10 GW
Posts: 4468
Joined: Dec 15, 2010 11:14 am
Location: Upstate-Western-Southern Tier NY. USA

Re: Forum software upgrade on 11/17/2017 technical issues thread

Post by DrkAngel » Nov 22, 2017 11:51 pm

Hmm, ES Wiki does work with IE, just stopped working with new Firefox Quantum 57.0. ... ?
A little learning is a dangerous thing;
Drink deep, or taste not the Pierian spring:
There, shallow draughts intoxicate the brain,
And drinking largely, sobers us again.

I enjoy enlightening ... and enlivening the spirit of the innovators.

New & Improved - Acronym Definitions

Index - Homemade Battery Packs - Updated - November 2015

EBike Toolbox - Bargains! $

Endless Sphere Wiki

User avatar
neptronix
100 GW
100 GW
Posts: 13063
Joined: Jun 15, 2010 5:56 pm
Location: People's republic of Petrolistan ( Utah, USA )
Contact:

Re: Forum software upgrade on 11/17/2017 technical issues thread

Post by neptronix » Nov 22, 2017 11:59 pm

FYI, i believe we have found the source of the performance issues.

I have done a trace on what database calls are happening and noticed that phpbb3.2 does database writes at what looks like a much higher rate / intensity than the oldschool phpbb 3.0.x.

Because of this, we hit a IO rate limit with amazon, so we are periodically getting throttled down. The symptoms looked exactly like some kind of DDOS attack early on, because the server and database would perform great for a few hours... then randomly stop responding and/or reboot. This made diagnosis of the problem rather difficult.

Around 3:30-4am PST, i am taking the system offline for about an hour to move the database over to a quicker type.
I think this will do the trick. If it does not, i'm still on the case.

Thanks for being patient with me.. i am determined to make this system as snappy... if not snappier than it was before. :)
My first major build: 8T MAC motor on a Trek 4500.
The new all-arounder: Leafmotor 1500w on a Turner O2 full suspension.
The wheelie machine: 20" Rear Magic Pie II on a Trek 4300 MTB

"The best time to plant a tree was 20 years ago. The second best time is now."- Chinese Proverb

e-beach
10 MW
10 MW
Posts: 2281
Joined: Jan 10, 2012 9:48 pm
Location: Any Los Angeles area beach I am at. Or Santa Monica or possibly the south bay beaches.

Re: Forum software upgrade on 11/17/2017 technical issues thread

Post by e-beach » Nov 23, 2017 12:06 am

I waited around until ES was back on line, but it now loads really slowly. Sometimes taking close to 60 seconds to load. ES never took that long to load before. I am running Firefox Quantum 57.0 with (supposed) 100 meg + download speed. It is certainly slower now then before.
Favorite Quote: "This is L.A., sugar. There is no 'over the top." --- Chris Erskine

Current build: Liahona w/ cheap front suspension and suspension seat post. Yescomusa 36v 800w generic front hub motor. 15ah Headway triangle mounted pack. Tronsung 30 amp,

Previous Build:1992 Trek Antelope 800 - Bone Crusher (no suspension) - Yescomusa 800 watt 36 volt front wheel kit. Don't do it! Get suspension!!!

User avatar
amberwolf
100 GW
100 GW
Posts: 23948
Joined: Aug 17, 2009 6:43 am
Location: Phoenix, AZ, USA, Earth, Sol, Local Bubble, Orion Arm, Milky Way, Local Group
Contact:

Re: Forum software upgrade on 11/17/2017 technical issues thread

Post by amberwolf » Nov 23, 2017 12:30 am

Well, it's still several hours before he's going to do the work to fix that, so the slowness is to be expected. ;)

User avatar
neptronix
100 GW
100 GW
Posts: 13063
Joined: Jun 15, 2010 5:56 pm
Location: People's republic of Petrolistan ( Utah, USA )
Contact:

Re: Forum software upgrade on 11/17/2017 technical issues thread

Post by neptronix » Nov 23, 2017 6:29 am

Bang! the upgrade/maintenance is finished.
Let's give this 24 hours and see how it goes.
My first major build: 8T MAC motor on a Trek 4500.
The new all-arounder: Leafmotor 1500w on a Turner O2 full suspension.
The wheelie machine: 20" Rear Magic Pie II on a Trek 4300 MTB

"The best time to plant a tree was 20 years ago. The second best time is now."- Chinese Proverb

e-beach
10 MW
10 MW
Posts: 2281
Joined: Jan 10, 2012 9:48 pm
Location: Any Los Angeles area beach I am at. Or Santa Monica or possibly the south bay beaches.

Re: Forum software upgrade on 11/17/2017 technical issues thread

Post by e-beach » Nov 23, 2017 2:10 pm

It seem to be faster then it was.....a couple of seconds to load vs one or two minutes like it was after the upgrade. Thank you for the faster ES!

:D
Favorite Quote: "This is L.A., sugar. There is no 'over the top." --- Chris Erskine

Current build: Liahona w/ cheap front suspension and suspension seat post. Yescomusa 36v 800w generic front hub motor. 15ah Headway triangle mounted pack. Tronsung 30 amp,

Previous Build:1992 Trek Antelope 800 - Bone Crusher (no suspension) - Yescomusa 800 watt 36 volt front wheel kit. Don't do it! Get suspension!!!

User avatar
fechter
100 GW
100 GW
Posts: 11822
Joined: Dec 31, 2006 3:23 pm
Location: California Bay Area, USA

Re: Forum software upgrade on 11/17/2017 technical issues thread

Post by fechter » Nov 23, 2017 6:24 pm

Speed seems to be back to near normal. Good work!
"One test is worth a thousand opinions"

User avatar
teklektik
1.21 GW
1.21 GW
Posts: 3524
Joined: Mar 26, 2011 1:15 pm
Location: CT, USA

Re: Forum software upgrade on 11/17/2017 technical issues thread

Post by teklektik » Nov 23, 2017 6:50 pm

Yep -speed is much improved! :D
Visit Grin Technologies at www.ebikes.ca
Build Thread: 2WD Yuba Mundo V4

Post Reply