Move to XenForo software

This section will contain posting guidelines for the forum along with any other information that is relevant to this forum.
User avatar
neptronix   100 GW

100 GW
Posts: 15852
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Sep 14 2021 2:21pm

Thank you.

The last test is being run. Fingers are crossed.
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Heavy duty Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

User avatar
nicobie   1 GW

1 GW
Posts: 3287
Joined: Aug 07 2008 1:48pm
Location: Central Coast CA, USA

Re: Move to XenForo software

Post by nicobie » Sep 14 2021 2:40pm

🤞...
Ask for it by name.

my eTownie build thread: http://endless-sphere.com/forums/viewto ... =6&t=23701
Magazine article https://www.electricbike.com/nicks-etownie/

User avatar
neptronix   100 GW

100 GW
Posts: 15852
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Sep 17 2021 3:07pm

Fark. The testing is only halfway done. WTF :roll:
It turns out using exiftool to detect jpeg rotation puts a >3x penalty on the speed of the process. :evil:
Process completion time is more like a week, instead of 29 hours. :shock:

Time to optimize further.
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Heavy duty Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

User avatar
neptronix   100 GW

100 GW
Posts: 15852
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Sep 20 2021 5:46pm

So i had to write a poor man's profiler to get to the bottom of this.

GD ( for writing jpegs to BMPs ) and exiftool were using a majority of the cpu time... both unlikely suspects!
imagetender performance problem 2.png
I found that GD's imagebmp ( writes a bmp file ) takes 20x more time to using imagepng write than a png :lol:
Obviously GD's code for writing BMP is VERY un-optimized... improved disk speed can't make up a 20x different in performance. :lol:

I also changed exiftool to run conditionally and perform a check before copying the jpeg rotation data.. thus cut down exiftool use a lot.
imagetender performance problem 4.png
Now the load is dominated by cjpeg ( mozilla's tricked out jpeg encoder ), which is exactly what i wanted to see!

I'm amazed... the system is twice as fast, so the run through the images should be around 14 hours, not 29, and certainly not a week :lol:


Now that the performance crisis is over ( most of the high hanging fruit has been picked ), i just need to rule out that this is an answer to the rotating jpegs problem and then i should be able to move on :bolt: :es: :bolt:
You do not have the required permissions to view the files attached to this post.
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Heavy duty Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

User avatar
neptronix   100 GW

100 GW
Posts: 15852
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Sep 27 2021 7:55pm

Well it looks like imageTender finally ran both without error and without any sign of the jpeg rotation problem :bigthumb:
Looks like the final process is 18 hours instead of the original 29 as well.
imagetender final.png
Filesize checks out. Two temp files hanging out. Same number of 0 byte files. Etc.
imagetender.png
The 'adding pictures?' thread was used as a comparison for older images.
viewtopic.php?f=1&t=14748&start=25

ImageTender was specifically designed to not touch corrupt files. It handles the era of the most corrupt files ( 2010 ) perfectly.
I used the early posts on the stealth owners thread to go through this era.
imagetender corruption handling.png
I'd say i'm finally confident in rolling this out on the main server and then moving on to the actual Xenforo tasks now that we've massively reduced the time to xenforo import. :)


Meanwhile, mr kahuna has been working on a wiki solution, and everyone else tasked to work with php hasn't made motion. I intend to pick up the slack since those tasks are many times easier than this one.
You do not have the required permissions to view the files attached to this post.
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Heavy duty Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

User avatar
neptronix   100 GW

100 GW
Posts: 15852
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Oct 05 2021 5:10pm

It may be a few weeks until i can get started on the ES project again. But the rest is easy and the end is in sight.

Meanwhile in the office, i am setting up development infrastructure for my company as a whole so that i can hire remote workers and give them seamless access to a perfect copy of whatever server - without giving them direct access to live servers.

ES gets to hitch a ride on this infrastructure, and we will truly be enabled to make the forum a worldwide open source project. :mrgreen:

I would like to give Mr Kahuna a public shoutout for providing inspiration and wisdom recently, also making some progress on getting the ES Wiki 2.0 going already.

Ttyl.
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Heavy duty Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

Redsea Rider   10 mW

10 mW
Posts: 22
Joined: Jul 17 2020 7:06pm
Contact:

Re: Move to XenForo software

Post by Redsea Rider » Oct 05 2021 11:02pm

Sounds superb!
great job keep it up!

thanks I really appreciate your time and effort Neps.
Massawa
N.A and E.U Director
Business Development
sales@fardriver.io

User avatar
kauaicycler   100 W

100 W
Posts: 160
Joined: May 18 2019 1:37pm
Location: Cocoa, FL
Contact:

Re: Move to XenForo software

Post by kauaicycler » Oct 06 2021 10:09am

neptronix wrote:
Oct 05 2021 5:10pm

I would like to give Mr Kahuna a public shoutout for providing inspiration and wisdom recently, also making some progress on getting the ES Wiki 2.0 going already.

Ttyl.
Thanks! I'm happy I found a way to give back to this community. I'm excited to see what we create.
Build Thread: Meet Miss Mino'aka (Hawaiian for smile) Electra Lux 7D Fat Tire with a BBSHD

markz   100 GW

100 GW
Posts: 12144
Joined: Jan 09 2014 11:38pm

Re: Move to XenForo software

Post by markz » Oct 11 2021 12:18am

Why is there more spammers these days?
It's probably them going through all the anti-spam security just to end up with 1-4 posts before getting the hammer.


neptronix wrote:
Oct 05 2021 5:10pm
-

User avatar
ZeroEm   10 MW

10 MW
Posts: 2002
Joined: May 03 2019 11:53am
Location: San Antonio, TX

Re: Move to XenForo software

Post by ZeroEm » Oct 11 2021 9:32am

Thought it was better the last few weeks?
by markz » Oct 11 2021 12:18am

Why is there more spammers these days?
It's probably them going through all the anti-spam security just to end up with 1-4 posts before getting the hammer.
2019 Performer E-Trike
2013 Nissan Leaf S 7 bars 331.5w/KM

markz   100 GW

100 GW
Posts: 12144
Joined: Jan 09 2014 11:38pm

Re: Move to XenForo software

Post by markz » Oct 11 2021 2:19pm

The spammers just want their poetry deleted.
Whats the use? I guess if it stays their words get a higher ranking in the search engines?

The only way around it is to have a waiting period when new users join, and to force them into a Welcome Thread where they have to post before moving on to posting in other threads.
I believe Nep talked about that and thats been squashed, but you are right its not really that bad, I just find theres a little more of it lately.
Altenwerth wrote:
Oct 11 2021 4:52am
They are fierce warriors whose movements and speech provide an amusing display of double trouble.

ZeroEm wrote:
Oct 11 2021 9:32am
Thought it was better the last few weeks?
by markz » Oct 11 2021 12:18am

Why is there more spammers these days?
It's probably them going through all the anti-spam security just to end up with 1-4 posts before getting the hammer.

User avatar
neptronix   100 GW

100 GW
Posts: 15852
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Oct 11 2021 4:03pm

Many phpbb anti spam plugins no longer work.. and spammers come in waves.
Our anti spam systems are simplistic and other options are not effective or feasible for moderator time.

..another reason why we're moving! Xenforo's got the anti spam features nailed down pretty good! i don't want our moderators wasting much time with that, nor the poetry being read for long. :)

PS
Thanks for the thanks, everyone.
I'm trying to clear out a good chunk of my mid-priority stuff so i can throw some serious time at ES. :thumb:
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Heavy duty Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

Post Reply