Move to XenForo software

This section will contain posting guidelines for the forum along with any other information that is relevant to this forum.
User avatar
neptronix   100 GW

100 GW
Posts: 15852
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Move to XenForo software

Post by neptronix » Feb 05 2021 2:50pm

Hey all.

An ongoing issue with phpbb's youtube embeds has lit a fire on my ass to start building a test site for ES 2.0 running XenForo instead of phpbb.

I am building the server today, will have the platform installed, and will start a test instance of our entire setup.. hopefully by EOD.
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Heavy duty Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

User avatar
neptronix   100 GW

100 GW
Posts: 15852
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Feb 05 2021 3:21pm

Test server up and running. :thumb:
server.png
You do not have the required permissions to view the files attached to this post.
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Heavy duty Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

User avatar
neptronix   100 GW

100 GW
Posts: 15852
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Feb 05 2021 5:05pm

Spent a number of hours getting a version of MySQL that works on both XenForo and phpBB.. fun times :lol:
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Heavy duty Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

User avatar
neptronix   100 GW

100 GW
Posts: 15852
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Feb 05 2021 6:20pm

Development copy of endless sphere finally working on new mysql.

Xenforo gettin' uploaded.
up.png
You do not have the required permissions to view the files attached to this post.
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Heavy duty Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

User avatar
neptronix   100 GW

100 GW
Posts: 15852
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Feb 05 2021 7:04pm

Why i hate AWS web hosting in one image:
slowdisk.png
Seeing write speeds average 12-15mb/sec on the most expensive and fast SSD they have.. with all the performance settings jacked up as far as they go. Occasionally i see peaks in the 100mb's for a second, and the CPU is averaging 50% load on both cores..

I've got a 7 gigabyte database to write to disk and this takes an hour or more.. i could have got this done faster on my linux desktop.

NOT the best hosting... but probably the most stable... WAY more stable and consistent than the digitalocean server we're on.. but damn, i want to run one of my own server farms one of these days.

Just waiting on the database crunching before i take a full disk backup and then hit the shiny button.
2021-02-05 16_54_11-XenForo 2.2.3 Patch 1 - Welcome _ XenForo.png
You do not have the required permissions to view the files attached to this post.
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Heavy duty Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

User avatar
neptronix   100 GW

100 GW
Posts: 15852
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Feb 05 2021 8:30pm

Got hit with the amazon disk speed throttle real bad.. 4 hours i'm still waiting on a nice database crunch..

Ah well, i'll be back on the task this weekend.
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Heavy duty Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

User avatar
nicobie   1 GW

1 GW
Posts: 3287
Joined: Aug 07 2008 1:48pm
Location: Central Coast CA, USA

Re: Move to XenForo software

Post by nicobie » Feb 06 2021 3:53pm

Sounds irritating as hell. I'm sure all the members will be thankful for the new features. I know I will.

Much thanks for your time spent on this necessary upgrade.
Ask for it by name.

my eTownie build thread: http://endless-sphere.com/forums/viewto ... =6&t=23701
Magazine article https://www.electricbike.com/nicks-etownie/

User avatar
neptronix   100 GW

100 GW
Posts: 15852
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Feb 06 2021 4:28pm

You're welcome.

Last night i encountered the usual massive MySQL write speed issues that cause all kinds of server operations to happen at a crawl.

I've taken the slow backup restore process down from 5 hours to 12 minutes after working on tuning MySQL all morning.
I've cut dozens of hours off the xenforo migrating, testing, etc process. Very 'stoked', as old surfers say on the central coast :mrgreen:
2021-02-06 13_59_49-Volumes _ EC2 Management Console.png
I'm truly shocked by the results. The high volume tune did not benefit my big server administration client nearly this much. It confirms to me that the database design of phpbb is far from optimal when operated at scale.

I wish i knew about this 5 years ago!!! unfortunately, information on the 'net about optimizing phpbb performance is lacking because most people run small forums that died off due to hacking, updates breaking things, etc.. so the collective knowledge never got to this point.
You do not have the required permissions to view the files attached to this post.
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Heavy duty Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

User avatar
neptronix   100 GW

100 GW
Posts: 15852
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Feb 08 2021 2:47pm

The project is a bit stalled as i take care of my clients today. But will continue in a few days.

Over the weekend, i did some work on my PHP framework, zerolith. It's a framework that's designed to plug into ANY system running on php, whether it has a framework or not. I am building some commercial software on it, and also using it to do things like log emails, debug messages, clean up databases, etc for many of my clients.

The process of migrating to XenForo, and the process of developing custom code that adds functionality to XenForo will benefit from this greatly.

This week, i created a debugger which will provide as a better aid than XenForo's built in debugger; since we're going to have LOTS of bugs up front, it'd be best to trace them down them as quickly as possible.
img.png
Anyway, more later.
You do not have the required permissions to view the files attached to this post.
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Heavy duty Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

User avatar
Dui, ni shuo de dui   10 kW

10 kW
Posts: 595
Joined: Jan 29 2016 3:21am

Re: Move to XenForo software

Post by Dui, ni shuo de dui » Feb 09 2021 2:42am

Thanks a lot for this hard work, I can't wait until the forum is being updated, I'm sure it will be awesome!

One stupid question maybe: do you know if we will lose the pictures? that's generally a big issue I saw on some other forums, sometimes after migration the pictures links won't work anymore, which is a huge problem for build threads and tutorials for example.

Again, thanks a lot for the great work and for maintaining this forum!
:bolt: New Build in Progress: Electric Roadster, 30 000W Kelly KLS72701, 24S2P A123 cells :bolt:
viewtopic.php?f=12&t=109246

:bolt: My electric Ninja 250 clone: 16 000W Sabvoton 72200, 23S2P A123 cells : :bolt:
viewtopic.php?f=12&t=90032

:bolt: My electric Scooter: 11 000W Sabvoton 72150, 72V 50Ah LiFePO4 cells: :bolt:
viewtopic.php?f=12&t=75912

User avatar
nicobie   1 GW

1 GW
Posts: 3287
Joined: Aug 07 2008 1:48pm
Location: Central Coast CA, USA

Re: Move to XenForo software

Post by nicobie » Feb 09 2021 1:01pm

How about old PM's? will they transfer?

The reason I ask is that I have a few old ones that have links in them I would like to keep. If I know that they will go away in the transfer, I will take the time to look them up and store them somewhere else.
Ask for it by name.

my eTownie build thread: http://endless-sphere.com/forums/viewto ... =6&t=23701
Magazine article https://www.electricbike.com/nicks-etownie/

User avatar
neptronix   100 GW

100 GW
Posts: 15852
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Feb 09 2021 1:34pm

Dui, ni shuo de dui wrote:
Feb 09 2021 2:42am
One stupid question maybe: do you know if we will lose the pictures? that's generally a big issue I saw on some other forums, sometimes after migration the pictures links won't work anymore, which is a huge problem for build threads and tutorials for example.

Again, thanks a lot for the great work and for maintaining this forum!
That's not a stupid question. Preserving images is of utmost importance. We have lost some images along the way also due to database and file system issues that are common with phpbb whenever you jostle any setting.

Before we move, i have some large scale data processing tasks to partake in to try to turn external hosted images into local ones, and recover whatever file pointers have been lost to database corruption in the past.
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Heavy duty Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

User avatar
neptronix   100 GW

100 GW
Posts: 15852
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Feb 09 2021 1:50pm

nicobie wrote:
Feb 09 2021 1:01pm
How about old PM's? will they transfer?
I have no idea yet. If they cannot be transferred, some way will have to be invented for you to list and read 'legacy PMs'.
We have hundreds of megabytes of PMs, so it's very important to preserve that.


As for the project status...

My client schedule this week isn't super packed, so there'll be another burst of activity this week on this. I'm also getting gammaray on task to work with me so we can tag team some things.

The goal for the end of the week is to have a working xenforo conversion of the forum, sans all the customizations. We'll come up with a list of what data transfers and what doesn't, and formulate a plan to handle that best.

If there is anyone here who would like to volunteer to help test these things, please raise your hand.
I would bet amberwolf would be willing, but i haven't asked him yet.
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Heavy duty Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

User avatar
nicobie   1 GW

1 GW
Posts: 3287
Joined: Aug 07 2008 1:48pm
Location: Central Coast CA, USA

Re: Move to XenForo software

Post by nicobie » Feb 09 2021 3:42pm

No worries! It will get done sooner or later.

I'd be happy to help out any way I can.

Nick
Ask for it by name.

my eTownie build thread: http://endless-sphere.com/forums/viewto ... =6&t=23701
Magazine article https://www.electricbike.com/nicks-etownie/

User avatar
neptronix   100 GW

100 GW
Posts: 15852
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Feb 09 2021 3:55pm

Sweet. Anyone else?

I'll come up on instructions for those volunteering for what to test, or you can simply free range if you prefer.
The more eyes on it, the better. And different eyes will see different things. :)
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Heavy duty Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

User avatar
Dauntless   100 GW

100 GW
Posts: 10113
Joined: May 29 2010 1:49am
Location: Coordinates: 33°52′48″N 117°55′43″W

Re: Move to XenForo software

Post by Dauntless » Feb 09 2021 4:54pm

neptronix wrote:
Feb 09 2021 3:55pm
I'll come up on instructions for those volunteering for what to test, or you can simply free range if you prefer.
The more eyes on it, the better. And different eyes will see different things. :)
Depends on the testing. I'm really not a programmer, but I could try out a beta site if that's what you mean.
Any sufficiently advanced technology is INDISTINGUISHABLE FROM MAGIC!
- Arthur C. Clarke

User avatar
neptronix   100 GW

100 GW
Posts: 15852
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Feb 09 2021 6:13pm

Dauntless wrote:
Feb 09 2021 4:54pm
Depends on the testing. I'm really not a programmer, but I could try out a beta site if that's what you mean.
That is what i mean, sorry for not being clear.

Basically i'll hand over a password to access ES 2.0 'beta' which will have a functional conversion of ES' content on it.
You'd need to poke around, and come up with a list of what's broken.

Once we have a list of things that are broken, it's my turn to fix 'em.
Then we do another round of poking around to confirm the fixes.
Then we do another round to confirm all the fun plugins we'll be installing don't have any goofy side effects.

Each round takes an hour or so, and you can do one round, two, three, whatever.
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Heavy duty Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

User avatar
neptronix   100 GW

100 GW
Posts: 15852
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Feb 10 2021 4:22pm

Applied some database optimizations to the current server to speed up dumping the DB and hopefully give us more stable/consistent performance on digitalocean while we're on here. Their slow IO has been plaguing our server since we moved there.

We had a little ~10 minute glitch while i did it.
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Heavy duty Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

User avatar
neptronix   100 GW

100 GW
Posts: 15852
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Feb 10 2021 4:44pm

Still getting a little disk i/o wait episodes but they're not as bad. OK, ES is stabilized. 8)
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Heavy duty Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

User avatar
kauaicycler   100 W

100 W
Posts: 160
Joined: May 18 2019 1:37pm
Location: Cocoa, FL
Contact:

Re: Move to XenForo software

Post by kauaicycler » Feb 11 2021 10:46am

I'm willing to beta test.
Build Thread: Meet Miss Mino'aka (Hawaiian for smile) Electra Lux 7D Fat Tire with a BBSHD

User avatar
neptronix   100 GW

100 GW
Posts: 15852
Joined: Jun 15 2010 5:56pm
Location: Utah, USA
Contact:

Re: Move to XenForo software

Post by neptronix » Feb 11 2021 2:42pm

kauaicycler wrote:
Feb 11 2021 10:46am
I'm willing to beta test.
Cool, thank you. :thumb:
"Love and compassion are necessities, not luxuries. Without them, humanity cannot survive." - Dalai Lama

My first major build: 1.6kW 8T MAC motor on a Trek 4500 MTB.
Monster MTB: Leafmotor 1500w @ 4kW on a Turner O2 full suspension.
The monster scooter: Heavy duty Cannondale semi recumbent - under construction.
Blue Dream: Maxaraya FS semi recumbent and high efficiency mid-drive - under construction.

User avatar
amberwolf   100 GW

100 GW
Posts: 32238
Joined: Aug 17 2009 6:43am
Location: Phoenix, AZ, USA, Earth, Sol, Local Bubble, Orion Arm, Milky Way, Local Group
Contact:

Re: Move to XenForo software

Post by amberwolf » Feb 14 2021 12:24am

When we move over, perhaps borrow this quote to stick in the header:
“ Let us stand on each other’s shoulders, not each other’s toes”
Dennis Allison

Maybe it would help with the attitude problem a few of the people have around here (mostly those that end up in OTD a lot, but there are others....) :)

User avatar
amberwolf   100 GW

100 GW
Posts: 32238
Joined: Aug 17 2009 6:43am
Location: Phoenix, AZ, USA, Earth, Sol, Local Bubble, Orion Arm, Milky Way, Local Group
Contact:

Re: Move to XenForo software

Post by amberwolf » Feb 15 2021 11:53pm

Also, when the memberlist is migrated, I would recommend not migrating any zero-post members. This will remove a huge amount of members from the list, most of whom were spammers.

Can't really not migrate banned members, unless they are separated from their posts, because there are quite a few banned members who left behind a lot of useful posts. ;)

User avatar
Dauntless   100 GW

100 GW
Posts: 10113
Joined: May 29 2010 1:49am
Location: Coordinates: 33°52′48″N 117°55′43″W

Re: Move to XenForo software

Post by Dauntless » Feb 16 2021 4:42pm

If you don't migrate the banned members, aren't they then unbanned? Safe, Bluto, and more that people could fill in names for, unleashed all at once.

Who ya gonna call?

GHOST BUSTERS!
Any sufficiently advanced technology is INDISTINGUISHABLE FROM MAGIC!
- Arthur C. Clarke

Jaraldo   1 µW

1 µW
Posts: 1
Joined: Oct 17 2020 5:19pm

Re: Move to XenForo software

Post by Jaraldo » Feb 17 2021 12:35am

Probably an unusual topic for a first post, but as a new member that has referenced the forum extensively in the last few months, I thank you for your effort in updating it - and to everyone who provides the content/community. :)

A community I was part of a few years ago migrated from phpbb to Xenforo as well and I helped with bug testing. If there is anything I can do, let me know.

I'm not technically proficient in how Xenforo works on the back-end, but just relaying information I've observed based on inquires in this topic so far:
  • When the forums were being moved to Xenforo, the biggest effort/headache was getting all the uploaded files/images (there were 100'000s) to sync up with the topics they were posted in. For whatever reason all the topics 3 years or older didn't show the files/images that had been posted in them - this could just be due to the admin who wasn't experience with Xenforo.
  • The PMs from the old site were able to be migrated to the new site.
  • I think the banned users from the old forum had to manually have their settings altered on the Xenforo forum in order to be banned again. Iirc, it had something to do with the differences between how Xenforo and phpbb organize data differently. As that was a few years ago, maybe there is a better way to do that now.
Cheers :bolt:

Post Reply