The BuddyPress next generation API, the journey begins
-
(Non-programmers feel free to skip to the last paragraph for an assignment that anyone can help with.)
Improving, refactoring, and extending the BuddyPress API is going to be a long journey for sure. In fact, it will probably never end, but hopefully it will become much easier and more enjoyable to work with.
From my viewpoint, BuddyPress is at a major crossroads. The current API has reached its maximum capacity to be extended, while at the same time the demand for plugins is eclipsing itself on what seems like a weekly basis. The goal is to get ahead of the curve and open up BuddyPress plugin development to a new generation of developers.
Before I get into any details, I want to be clear about the parameters of the refactoring.
1. Maintain low barrier to entry for developers (and probably lower it).
2. Maintain backwards compatibility (within reason).
3. All decisions in regards to the next generation API will be open to public discussion.
The major goals of the refactoring are listed below in no particular order.
1. Solve long standing major issues that have been identified as API limitations.
2. Eliminate duplicate functionality that exists in multiple components and plugins.
3. Add missing core functionality that is deemed to be of a high value to the BuddyPress community and to plugin developers.
4. Make extending BuddyPress easier.
5. Improve API documentation.
Ok, so How do we do it?
BuddyPress has always been developed using the iterative model. Variations of that model are the most popular and arguably the most successful for open source software development, so I don’t see any reason to change. I happen to be a fan of iterative development, so I am biased, but this is open for discussion. Even though it is a bit late to start from scratch , I still want to use prototyping at least during the first few iterations of the next generation API so we can hopefully detect major flaws before they become too widespread and embedded in the codebase.
The stages of the iterative model.
1. Analysis
2. Design
3. Coding
4. Testing
5. Maintenance
These should be repeated for each iteration (or sub-iteration) of development. They should be applied any time there is a problem to solve, and each step should be made public with a reasonable amount of time given for public discussion before moving on to the next step.
And so we are at step one, Analysis. I don’t think analyzing BuddyPress is a one, two, or even 10 step process. There is a lot of work to get done, and it feels overwhelming, so lets start with one very simple question. Before we can do anything, we have to all generally agree on one thing.
What is BuddyPress?
-
In your new role as API guru, is there any way to coordinate the efforts of plugin developers and core developers, in terms of letting plugin developers know what’s worth pursuing? How do you determine what goes in the core and what is plugin territory?
For example, I just read that “Inappropriate content flagging” is a “Potential Future BuddyPress Feature” for 1.4+ (https://buddypress.org/about/roadmap/) and now I’m wondering how that jibes with Report/Ignore plugin that may be developed as part of GSoC.
The need for an “advanced” BP search gets discussed frequently in the forums, so would that be a core function or should it be up to plugin developers?
I’ve made two contributions of code that extends BP Forums – a “posts since last visit” tracker and now a “BP Forum Move Topic” function that I will expand to Forum Topic Split and Topic Merge – but before I start extending that, or start another project, how do I know if I am duplicating effort that eventually will be part of the BP core?
Oh, and although every developer I’ve ever known has hated doing it, one of the single most important things for the whole project would be a comprehensive documentation project, well organized, with code samples and snippets.
Not going to hold my breath on that one, in part because I suppose it might feel like wasted effort to some extent until BP is fully mature (it’s getting close, though, I think).
Devs, you certainly know that Google will use next site speed for web search ranking ?.
http://googlewebmastercentral.blogspot.com/2010/04/using-site-speed-in-web-search-ranking.html
Probably not the best news for wp
There are many pros and cons to coordinating an effort like that. Development of this project is very organic, and developers come and go frequently. It would be bad if there is a specific functionality that everyone has marked as a high prioroty, and we are all sitting around waiting for a developer to get it done, and then he disappears.
The best advice I can give is to poll the community before you invest in a plugin to see if anyone else is working on the same thing. If you are working on a plugin, make it’s development as public as possible so that if the functionality it provides is needed for the core, the community is aware of it and can have a discussion about whether your plugin should be merged.
I think generally though, the future of BuddyPress will prove to be plugins. There are going to be solid core features to get people up and running quickly, and an API that allows for more advanced extensions without as many headaches.
The next generation API code will be well documented with PHPDoc (assuming everyone is cool with that), and I hope that we can have a section of the codex where the auto generated documentation for each release can be accessed for easy reference. Once the code is documented it will be much easier to recruit people to contribute to the BuddyPress codex.
Going to attempt to get this thread back on track with a summary of what I am seeing so far.
The hot keywords in no particular order:
social networking
community
niche
plugin
extend/extensible
framework
There seems to be a general consensus that BuddyPress is an application that allows even novice techies to easily add a social network or community component to their WP site. It provides the most essential tools for achieving this “right out of the box”. Since everyone is marketing their site to their niche, BuddyPress needs to be configurable and flexible enough to allow them to pick and choose what features are right for them and to select those features from a deep directory of solid plugins.
I think that if the low level core API of BuddyPress is strong and extensible, then ‘what BuddyPress is’ will mean different things to different people, depending on why they use it.
In a land before time, BuddyPress was actually intended to be several separate plugins, but having different components with different version numbers and dealing with the dependencies would have been a nightmare.
Now though, watching plugins merge and adapt, it doesn’t seem that unlikely.
Imagine if when you DL BuddyPress, you’re just DLing BuddyPress core, and the install routine lets you pick which additional components you might want for your social network, and it puts them in the BuddyPress directory. Not everyone needs friends, but maybe they want to follow each other. Different plugins for almost the same thing. Users should be walked through the component picking process to be able to shape their social network in a matter of minutes.
I believe the component API should be built for rapid deployment of new components. It should take care of the things we currently spend hours trying to dance around. I.E. component registration, template inclusion, URL rewriting, GET/POSTing, and it needs to give each new component its own API at the same time, so that we can easily communicate between these components and make them talk to each other in a universal way.
Then BuddyPress will become less of a “friends/activity/groups/profiles/pm” plugin, and more of a “sum of its parts” plugin.
So, the API would need to be fairly self aware, and include setters/getters/checkers for just about every routine there is for them to possibly do.
We had a ticket open about 4 months ago with a transitional API that I started for a general BP_Component class. It wasn’t much, but maybe a start?
I don’t mean to side-track the nature of this conversation (maybe I’m not), but as a developer in Higher-Ed arena I see a lot of potential for BP. There are a lot of buzz-words kicking around for the concepts of “open learning”: personal learning environments (PLE), communities of practice (COP), and these can be niche and community driven. I’m definitely in support of any ePortfolio endeavors as well. I would see these as being a series of plugins that rely on BP API, so that: Gradebook, Courses, DropBox could integrate with a user’s Activity Stream. Of course, granular levels of object permissions is ideal for users and admins. Great conversation!
You and I are on exactly the same page.
I think it would be cool if there was a BP core release and/or a BP “with goodies” release that shipped with maybe the top 20 plugins that are up to snuff code-wise already bundled with it but not activated.
I vaguely recall that ticket, and will dig it up to see what angle you took to solve the base component puzzle.
So I tried to explain this to my father…who can barely use email.
Pretend you’re in a room with 1,000 people that you don’t know. Now, you might end up making friends with 10 of them, and doing business with 2. The problem is that it would take you a week of shaking hands and starting conversations to find those connections.
Social Networks/Communities…like Buddypress/Facebook/LinkedIn etc. turn that week into 10 minutes.
To me, improved community-wide searching and easy navigation by profile fields (can someone please load in locations?) are the most important areas where BP is clunky and needs improvement. We need to focus on turning that 10-minutes into 5, while being ever-more-user-friendly and intuitive in the process.
The trend is that all of these networks will share information with one another, so much so that, in time, walls will be torn down and eventually, “Social Networks” will be a big mosh of everything…made up of smaller topic-focused communities like each of ours.
BP gives each of us a chance to create a niche within this great mosh. We shouldn’t try to BE the mosh, but simply be the most efficient and user-friendly communities within it.
@gregfielding – you are getting closer to a vision I have of the “net” – Imagine each person has their own “buddypress” and their friends have theirs – all on separate servers – friends are “linked” across the net via BP – my own activity feeds my friends canonically on their “what are my friends up to” page – my groups can be joined by my friends, but they do not have to be members, just request membership and wha-la they are members of my group – I can send email to the group – each one is separate, I can post updates which send email – etc.
Basically, you completely do away with the behemouth server farms by enabling creation of lite BP cells that are all linked together. How do you make that happen?
What JJJ said about it technicaly. What Andrea_r said about Facebook. I’ll add what I think as an admin.
BP is a stream for me as a user and not a techie. I need to be able to tweak everything that shows up in the stream in a way so that the community can expand. We’ve got 250 users now at a closed community with about 20% really active users. I’d like to grow up to at least 1000 users but I’m afraid of what will become of the Stream. I’ve got 300 very active friends over at Facebook and it makes me want to hide away from it. I mostly never go there since it’s too much to follow.
Also apart from the Stream there needs to be containers for all things for part time users. They should be able to follow the community even if they don’t take a peek every day.
@JJJ I’m not so sure the idea of a BP-core plus add-ons is the right approach. To me, the big question becomes where do you draw the line?
As for bundling plugins, please don’t do that. “Hello Dolly” still annoys me and I still have to keep deleting it. I don’t want to see a whole bunch of plugins show up in my install. A better approach is to have a much more user friendly plugin repository. I’m anxiously waiting to see what Andy churns out for that because my biggest gripe now is there is no easy way to know what is supported/recommended.
So following the plugin bunny trail that we’re on I’m going to throw my shoe out there.
BP is/was a group of plugins. When I install BP I’m not actually installing one plugin. I’m actually installing 7: Activity, Blog Tracking, bbPress Forums, Friends, Groups, Private Messaging, and Extended Profiles.
If BP can provide the best, fastest, most extensible core hooks and loops, then let the plugin developers code and extend to their hearts desire.
if (BP Core == diving board)
plugin authors = the bells and whistles of BP
The way I see it, there should be 3 classes of plugins.
1. The Core BP plugins (i.e. currently the 7 components… this list will increase, such as Basic Privacy).
If you’re not running all or most of these, then you’re not really running BP.
The BP is officially “in charge” of keeping these plugins updated.
2. The Senior Plugins (e.g. Media-Albums, Oembed for Activity Streams, BP SEO, Enhanced Privacy etc.)
These are plugins that almost 65% (or some other number) of sites will end up using but we’re not sure that they will.
BP officially endorses and most likely will use their paid developers to keep these plugins functional.
3. The Normal Plugins (e.g. GEO Locations, Extended Profiles, Gifts, etc.)
Plugins that authors create for different purposes. While many sites use them, they don’t have the same widespread use and support as the Senior Plugins.
BP has no official position with respect to these plugins.
Currently only category 1 and 3 exist. There is no category 2.
BuddyPress can be utlized in different ways for different projects … a social networking tool … an organizational tool … a management tool … and so much more. .
Thank you for BuddyPress
Any programmers out there who want to contribute to the analysis can follow the progress on the Trac wiki here https://trac.buddypress.org/wiki/NextGenApi
InactiveThe only thing BP should be focusing on is server load. BP php is extremely inefficient. Currently the only way to run BP is with a dedicated server (even with a single install). If you don’t believe this you don’t have enough traffic to make BP worthwhile in the first place.
There also needs to be some sort of default caching system in place. If phpbb can do it BP should be able to as well.
I would also recommend all plugins have to go through some sort of vetting before listed in the official WP/BP directory. Don’t get me wrong everyones hard work i s very much appreciated, but there is some bad code floating around out there that needs some sort of ground rules. There is nothing wrong with releasing “beta” plugins but there should also be some sort of stamp of approval for stable releases.
dennis_h: https://codex.buddypress.org/getting-started/improving-performance/
95% of users don’t need caching or a dedicated server because it would be overkill for the traffic they get. You are obviously in the 5% and should be implementing caching.
2. The Senior Plugins (e.g. Media-Albums, Oembed for Activity Streams, BP SEO, Enhanced Privacy etc.)
These are plugins that almost 65% (or some other number) of sites will end up using but we’re not sure that they will.
BP officially endorses and most likely will use their paid developers to keep these plugins functional.
I’d like to comment that the only developer who gets paid to work on BuddyPress is Andy Peatling. Everyone else contributes; some are consultants or developers or people who create themes, but that’s all on their own back.
@dennis_h
Starting with 1.4 when we begin to improve the API, improving the code for efficiency is going to be taken seriously.
That being said, I don’t think its fair to make a blanket like “BP php is extremely inefficient” without qualifying it. If you want to help improve the code, perhaps start a new thread and cite some specific examples of what code you think should be improved.
As far as the plugin repo goes, I would give similar advice, except it would probably be better to contact the authors directly first.
That being said, I don’t think its fair to make a blanket like “BP php is extremely inefficient” without qualifying it.
This could be an example of what dennis_h is talking about?
For me it goes back to the point that development now should focus on making the core rock solid, including security, privacy, member management, etc., before adding any more features.
I don’t think http requests to Gravatar qualify the statement “BP is extremely inefficient”. If you don’t want the extra requests the filter the Gravatars out. This is nothing to do with making the “core rock solid”.
Inactive@andy We have put in place most of the steps in that article and have seen some results, but thats kind of the point I was making. To really put those measures in place you need a dedicated server.
@MrMaz Maybe “BP php is extremely inefficient” was a little wide-ranging, but it was a phrase that came up over and over again in discussions with several hosting companies. For example, a senior tech at Blue Host/Host Monster told me every BP install they host (and they host over 1 million domains) is throttled (if users find their site slow this may be why).
In order for BP to be the next killer CMS I believe it should be able to be run on a shared server. To get the kind of plugin support and wide variety of quality themes BP needs to get those users that are either unwilling or unable to pay for a dedicated server.
PS
My current skill set is limited to testing and reporting back. That’s all I’m doing here, reporting what we experienced and what we where told. I’ll try to get better details.
I should also note I love BP. It’s a huge step forward and we will continue to use it and grow with it. I am only trying to offer feedback to help that growth. Thanks to everyone for their hard work that got us this far.
One more note. We have around 4000 members and are adding about 50 per day, so we would need a dedicated server regardless, but having a tighter BP would of course still be a big help.
@dennis_h
It would be great if you could get some useful feedback from a hosting company and share your findings so the issues can be addressed.
the api eh? (a large percentage of shared hosts are poor quality, we’ll never be able to fix that) my take is that at it’s core, it’s “the missing extended user profile framework for WordPress”. It would, in my feeble opinion act primarily as a framework api for extending those profiles. Some of the issues and limitations are really presented by wordpress and mu.. not Buddypress and in much the same way that bp extended the wordpress user profiles/functions we should tackle those head on as had been done with the improvements in the BP1.2 registration. There are infinite ways to build on those relational database fields. Focus exclusively on bp_xprofile _fields, and _data. It’s not that I’m anti-activity stream and the terms social network, life stream or whatever just seem to incorporate user profiles into some sort of relational scheme, be it friends, followers, ‘like’-ing, sharing or chronological micro blogging. Extending the stability, functionality, and interactivity of each component is the goal, but I’d prefer to see no dependency among core elements with one another. It’s fairly simple to build out a “social network” in other popular content management systems by extending the user database fields and in my day to day practice, I typically use only about 20% of the Buddypress core components for a WordPress project. Even though the hype is in the activity stream and the extensions for it (more facebook or twitter like), I side with @stwc in that “It also means that the platform has to have a robust set of tools for the administrator and moderators of the community “. A community (or ‘social network’) is just a set of users and to me, key items are a small footprint on the database and tight the integration with the existing wordpress user roles, permissions, registration and management. The api should ideally be flexible enough to accommodate any possible relational data between user profiles that a plugin author may dream of. I think the core integration between user profiles and the activity/blogs/groups/forums/friends illustrate the foundation of the api and should be as unified, consistent, and simple as possible with very little dependency or overlap in functionality. This would be impetus for creating a more standardized way to interact with the xprofile and be a good foundation for a solid api. lastly, thanks @mrmaz for starting the thread, being pro-active, getting the trac and api.buddypress and generally illustrating the potential of a solid api in regards to how well your links plugin and others can interact with the core.
ps.. death to PHP4 and fsck backwards compat
In case anyone hasn’t come across it yet, there is a new sub-blog entirely dedicated to the future of the BuddyPress API.
Please check it out! https://api.buddypress.org/
- The topic ‘The BuddyPress next generation API, the journey begins’ is closed to new replies.