Move to XenForo software

This section will contain posting guidelines for the forum along with any other information that is relevant to this forum.
Post Reply
User avatar
neptronix   100 GW

100 GW
Posts: 16341
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Sep 14 2021 2:21pm

Thank you.

The last test is being run. Fingers are crossed.
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

User avatar
nicobie   1.21 GW

1.21 GW
Posts: 3564
Joined: Aug 07 2008 1:48pm
Location: Central Coast CA, USA

Re: Move to XenForo software

Post by nicobie » Sep 14 2021 2:40pm

🤞...
Ask for it by name.

my eTownie build thread: http://endless-sphere.com/forums/viewto ... =6&t=23701
Magazine article https://www.electricbike.com/nicks-etownie/

User avatar
neptronix   100 GW

100 GW
Posts: 16341
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Sep 17 2021 3:07pm

Fark. The testing is only halfway done. WTF :roll:
It turns out using exiftool to detect jpeg rotation puts a >3x penalty on the speed of the process. :evil:
Process completion time is more like a week, instead of 29 hours. :shock:

Time to optimize further.
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

User avatar
neptronix   100 GW

100 GW
Posts: 16341
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Sep 20 2021 5:46pm

So i had to write a poor man's profiler to get to the bottom of this.

GD ( for writing jpegs to BMPs ) and exiftool were using a majority of the cpu time... both unlikely suspects!
imagetender performance problem 2.png
I found that GD's imagebmp ( writes a bmp file ) takes 20x more time to using imagepng write than a png :lol:
Obviously GD's code for writing BMP is VERY un-optimized... improved disk speed can't make up a 20x different in performance. :lol:

I also changed exiftool to run conditionally and perform a check before copying the jpeg rotation data.. thus cut down exiftool use a lot.
imagetender performance problem 4.png
Now the load is dominated by cjpeg ( mozilla's tricked out jpeg encoder ), which is exactly what i wanted to see!

I'm amazed... the system is twice as fast, so the run through the images should be around 14 hours, not 29, and certainly not a week :lol:


Now that the performance crisis is over ( most of the high hanging fruit has been picked ), i just need to rule out that this is an answer to the rotating jpegs problem and then i should be able to move on :bolt: :es: :bolt:
You do not have the required permissions to view the files attached to this post.
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

User avatar
neptronix   100 GW

100 GW
Posts: 16341
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Sep 27 2021 7:55pm

Well it looks like imageTender finally ran both without error and without any sign of the jpeg rotation problem :bigthumb:
Looks like the final process is 18 hours instead of the original 29 as well.
imagetender final.png
Filesize checks out. Two temp files hanging out. Same number of 0 byte files. Etc.
imagetender.png
The 'adding pictures?' thread was used as a comparison for older images.
viewtopic.php?f=1&t=14748&start=25

ImageTender was specifically designed to not touch corrupt files. It handles the era of the most corrupt files ( 2010 ) perfectly.
I used the early posts on the stealth owners thread to go through this era.
imagetender corruption handling.png
I'd say i'm finally confident in rolling this out on the main server and then moving on to the actual Xenforo tasks now that we've massively reduced the time to xenforo import. :)


Meanwhile, mr kahuna has been working on a wiki solution, and everyone else tasked to work with php hasn't made motion. I intend to pick up the slack since those tasks are many times easier than this one.
You do not have the required permissions to view the files attached to this post.
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

User avatar
neptronix   100 GW

100 GW
Posts: 16341
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Oct 05 2021 5:10pm

It may be a few weeks until i can get started on the ES project again. But the rest is easy and the end is in sight.

Meanwhile in the office, i am setting up development infrastructure for my company as a whole so that i can hire remote workers and give them seamless access to a perfect copy of whatever server - without giving them direct access to live servers.

ES gets to hitch a ride on this infrastructure, and we will truly be enabled to make the forum a worldwide open source project. :mrgreen:

I would like to give Mr Kahuna a public shoutout for providing inspiration and wisdom recently, also making some progress on getting the ES Wiki 2.0 going already.

Ttyl.
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

Redsea Rider   100 mW

100 mW
Posts: 39
Joined: Jul 17 2020 7:06pm
Contact:

Re: Move to XenForo software

Post by Redsea Rider » Oct 05 2021 11:02pm

Sounds superb!
great job keep it up!

thanks I really appreciate your time and effort Neps.
Driving the future forward,

Massawa Lawson
Sales Director
Nanjing Far Drive
USA EU WORLDWIDE
sales@fardriver.io

User avatar
kauaicycler   100 W

100 W
Posts: 167
Joined: May 18 2019 1:37pm
Location: Cocoa, FL
Contact:

Re: Move to XenForo software

Post by kauaicycler » Oct 06 2021 10:09am

neptronix wrote:
Oct 05 2021 5:10pm

I would like to give Mr Kahuna a public shoutout for providing inspiration and wisdom recently, also making some progress on getting the ES Wiki 2.0 going already.

Ttyl.
Thanks! I'm happy I found a way to give back to this community. I'm excited to see what we create.
Build Thread: Meet Miss Mino'aka (Hawaiian for smile) Electra Lux 7D Fat Tire with a BBSHD

User avatar
markz   100 GW

100 GW
Posts: 12306
Joined: Jan 09 2014 11:38pm
Location: Canada and the USA

Re: Move to XenForo software

Post by markz » Oct 11 2021 12:18am

Why is there more spammers these days?
It's probably them going through all the anti-spam security just to end up with 1-4 posts before getting the hammer.


neptronix wrote:
Oct 05 2021 5:10pm
-

User avatar
ZeroEm   10 MW

10 MW
Posts: 2334
Joined: May 03 2019 11:53am
Location: San Antonio, TX

Re: Move to XenForo software

Post by ZeroEm » Oct 11 2021 9:32am

Thought it was better the last few weeks?
by markz » Oct 11 2021 12:18am

Why is there more spammers these days?
It's probably them going through all the anti-spam security just to end up with 1-4 posts before getting the hammer.
2019 Performer E-Trike
2013 Nissan Leaf S 7 bars 331.5w/KM

User avatar
markz   100 GW

100 GW
Posts: 12306
Joined: Jan 09 2014 11:38pm
Location: Canada and the USA

Re: Move to XenForo software

Post by markz » Oct 11 2021 2:19pm

The spammers just want their poetry deleted.
Whats the use? I guess if it stays their words get a higher ranking in the search engines?

The only way around it is to have a waiting period when new users join, and to force them into a Welcome Thread where they have to post before moving on to posting in other threads.
I believe Nep talked about that and thats been squashed, but you are right its not really that bad, I just find theres a little more of it lately.
Altenwerth wrote:
Oct 11 2021 4:52am
They are fierce warriors whose movements and speech provide an amusing display of double trouble.

ZeroEm wrote:
Oct 11 2021 9:32am
Thought it was better the last few weeks?
by markz » Oct 11 2021 12:18am

Why is there more spammers these days?
It's probably them going through all the anti-spam security just to end up with 1-4 posts before getting the hammer.

User avatar
neptronix   100 GW

100 GW
Posts: 16341
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Oct 11 2021 4:03pm

Many phpbb anti spam plugins no longer work.. and spammers come in waves.
Our anti spam systems are simplistic and other options are not effective or feasible for moderator time.

..another reason why we're moving! Xenforo's got the anti spam features nailed down pretty good! i don't want our moderators wasting much time with that, nor the poetry being read for long. :)

PS
Thanks for the thanks, everyone.
I'm trying to clear out a good chunk of my mid-priority stuff so i can throw some serious time at ES. :thumb:
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

User avatar
markz   100 GW

100 GW
Posts: 12306
Joined: Jan 09 2014 11:38pm
Location: Canada and the USA

Re: Move to XenForo software

Post by markz » Oct 23 2021 6:42pm

Is there a quick links in the new XenForo, in phpBB there is a quick links next to FAQ near the top header, have an additional "Your Threads" along with "Your Posts"

User avatar
neptronix   100 GW

100 GW
Posts: 16341
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Oct 27 2021 8:55pm

That's a bare minimum feature and if it doesn't have it, i'll hack it in :)
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

User avatar
neptronix   100 GW

100 GW
Posts: 16341
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Nov 01 2021 12:31pm

ImageTender is being deployed today. Expect a little mayhem, but do know that i have backups. :)
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

User avatar
neptronix   100 GW

100 GW
Posts: 16341
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Nov 01 2021 1:37pm

OK.. there is a HUGE rift between the speed of AWS versus the 10th gen core i7 i tested the process on. About 3 times slower.
ESCompressor.png
The server will be in and out of consciousness in about 10 mins as i give it a little more zip.
You do not have the required permissions to view the files attached to this post.
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

User avatar
neptronix   100 GW

100 GW
Posts: 16341
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Nov 01 2021 5:03pm

This is unusual, but the code is performing as expected without bugs and doesn't need to be babysat. :lol:

I think you are not going to see any hiccups for the rest of the day while this thing is savagely beating on the CPU.
I overestimated the process time and think we'll have results by tomorrow.

In the meantime, you may end up seeing an image in a half-processed state.
You do not have the required permissions to view the files attached to this post.
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

User avatar
TheBeastie   10 MW

10 MW
Posts: 2085
Joined: Jul 28 2012 12:31am
Location: Melbourne Australia

Re: Move to XenForo software

Post by TheBeastie » Nov 02 2021 1:26am

neptronix wrote:
Nov 01 2021 1:37pm
OK.. there is a HUGE rift between the speed of AWS versus the 10th gen core i7 i tested the process on. About 3 times slower.

ESCompressor.png

The server will be in and out of consciousness in about 10 mins as i give it a little more zip.
You mean those AWS CPUs that Amazon made them selves for their AWS services?
Cool, yeah I read about those in tech news somewhere, it looks like just about every tech giant makes their own CPUS now if it doesn't require x86 binaries.
In this article they are called Graviton2, you can even see "an Amazon company" on the CPU in small writing.
https://www.anandtech.com/show/15578/cl ... el-and-amd
Image

Looks like processors are turning into a easily available commodity from a lot of sources instead of just something you could get from AMD/Intel/Sun like the old days.
Speed Kills Range, 10mph = 46 miles range, 20mph = 20 miles, 30mph = 8 miles rangehttps://goo.gl/1JNL53
Over Charging Kills ur battery bit.ly/1hzWKl4
Consider PAS as your only throttle https://goo.gl/Kg1F8F
Fuel-Cell is the ultimate battery coupled with 4th-gen Nuclear
https://goo.gl/TcKtHs https://goo.gl/ZhFFot https://goo.gl/gfa215
10 Square Miles of solar panels = 0.12GW average power! https://goo.gl/Ub1S39

User avatar
neptronix   100 GW

100 GW
Posts: 16341
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Nov 02 2021 7:39am

I think ARM processors are pretty cool, good performance per dollar, but haven't been produced with great clock speeds just yet.. many important time sucking processes on a site like ours are single threaded, so a small amount of very fast cores is needed... we stick with x86 for the time being.

Also, i'd like a precompiled binary package for everything and i'm not sure if i get that with an ARM version of a linux server distro. Haven't looked into that though.
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

User avatar
neptronix   100 GW

100 GW
Posts: 16341
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Nov 02 2021 3:55pm

Done. Freed up about 29gb of image pork. The disk was about to be full before. :bolt:
done.png
So glad it's done!! now.. to set up a fresh new dev system for the remaining work and testing some of the other prerequisite surgeries.
You do not have the required permissions to view the files attached to this post.
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

User avatar
neptronix   100 GW

100 GW
Posts: 16341
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Nov 10 2021 7:44pm

Okay, repairing the corrupt image files from the past and will then upload them back to the server.
After that, i'm refreshing the development machine for the volunteers to use.

I'm also setting up my first micro data center. Got a fixed IP out to the world and a secure location.
The ES Dev system hitches a ride for free.

We do have some cool shit planned and i'm sorry it's taken a while!
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

User avatar
nicobie   1.21 GW

1.21 GW
Posts: 3564
Joined: Aug 07 2008 1:48pm
Location: Central Coast CA, USA

Re: Move to XenForo software

Post by nicobie » Nov 10 2021 8:43pm

No worries!

Any progress is good. It's not like you're getting paid or under a deadline. 👍👍
Ask for it by name.

my eTownie build thread: http://endless-sphere.com/forums/viewto ... =6&t=23701
Magazine article https://www.electricbike.com/nicks-etownie/

User avatar
neptronix   100 GW

100 GW
Posts: 16341
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Nov 29 2021 5:44pm

Things are going well. I still have some things to clear off my plate before i get heavy into another ES round though.


Because the crypto miners had to buy full computers to get GPUs for a while, there's shit-tons of spare GPU-less and new machines being hawked on ebay. I bought ES + my company a mini Dell server with a i7-10700 for $460! and 1 year old!

It replaces it's 'baby bro' to the left that only had a core i5.
( baby bro was the dev box until i found out imageTender would take >40 hours running on it; i had to run imageTender on my own 10700 during testing!! and that was nowhere near fast enough! ).
img.jpg
It will be a great dev box for the $ and we need that 4.8ghz peak for crunching gigabytes of data for hours on end.

Should have it set up fairly soon and unleash the Kahuna on it to have him set up the new wiki ahead of the xenforo move.
You do not have the required permissions to view the files attached to this post.
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

User avatar
amberwolf   100 GW

100 GW
Posts: 33696
Joined: Aug 17 2009 6:43am
Location: Phoenix, AZ, USA, Earth, Sol, Local Bubble, Orion Arm, Milky Way, Local Group
Contact:

Re: Move to XenForo software

Post by amberwolf » Nov 30 2021 3:34am

Just curious: Harddisk, or SSD type? I've seen that SSDs seem to be way way faster for random access....


Also, unrelated, but is it possible to "move" a complete harddisk clone from an old Win10 laptop to a new drive on a different machine (like one of those you've got there), without having to reinstall Windows (just reactivate it or whatever) so that I don't have to reinstall and reconfigure a bajillion pieces of software, music plugins, etc? There arent' any install media, just a restore partition, which presumably only works on the laptop....

I ask because my laptop isn't quite sufficient anymore for what I want to do with music, and someday when I have a job again and enough money coming in I'd like to use a desktop for the music stuff for the expansion capabilities and multimonitor possiblities. (and the laptop is having more and more random issues with hardware and/or crashes over time).
If you found this advice helpful, supporting contributions are accepted here.

Beginning of a New Life


Give a listen to my music!

(More here, including experimental)
.

User avatar
kauaicycler   100 W

100 W
Posts: 167
Joined: May 18 2019 1:37pm
Location: Cocoa, FL
Contact:

Re: Move to XenForo software

Post by kauaicycler » Nov 30 2021 9:12am

neptronix wrote:
Nov 29 2021 5:44pm
unleash the Kahuna
:lol: :lol: :lol: :bigthumb:
Build Thread: Meet Miss Mino'aka (Hawaiian for smile) Electra Lux 7D Fat Tire with a BBSHD

Post Reply