URLs in posts, need expert help on a regex for finding them

[DarkPikachu]

as the title says, I need help designing a regex to identify urls outside of <a ...> tags.
(my plugin converts [ a] and [ url] tags to <a ...> before the code I have checks for unlinked URLs)

I'm not sure if a url like tcll5850.proboards.com works (no http(s) or www)... I don't pay too much attention to the behavior

here's what I have so far:
.replace(/(http(|s):\/\/|www.)(.+?)( |\n)/g,'<a href="$2" target=“_blank">$1$2</a>$3')

it gets anything starting with 'http(s)://' or 'www.' and ending with ' ' or '\n' and converts it into a link...

what would be even more helpful here is if I knew exactly how PB did it (in code).


could anyone help me out here??

EDIT:
oh heck, the link does work
yea I really need some help here :/
please and thanks

[DarkPikachu]

found this one on SO: stackoverflow.com/questions/3809401/what-is-a-good-regular-expression-to-match-a-url

/[-a-zA-Z0-9@:%_\+.~#?&//=]{2,256}\.[a-z]{2,4}\b(\/[-a-zA-Z0-9@:%_\+.~#?&//=]*)?/gi

not really sure how to deal with this one on a case of a replace() >.>
and regex101.com says it has an error...

EDIT: forget about regex101, it sucks
regexr.com/2s81g

EDIT2:
also got this one:
var isValidUrl = /^(??:https?|ftp):\/\/)(?:\S+(?::\S*)?@)?(??!10(?:\.\d{1,3}){3})(?!127(?:\.??\d{1,3}){3})(?!169\.254(?:\.\d{1,3}){2})(?!192\.168(?:\.\d{1,3}){2})(?!172\.(?:1[??6-9]|2\d|3[0-1])(?:\.\d{1,3}){2})(?:[1-9]\d?|1\d\d|2[01]\d|22[0-3])(?:\.(?:1?\d{1??,2}|2[0-4]\d|25[0-5])){2}(?:\.(?:[1-9]\d?|1\d\d|2[0-4]\d|25[0-4]))|(??:[a-z\u00??a1-\uffff0-9]+-?)*[a-z\u00a1-\uffff0-9]+)(?:\.(?:[a-z\u00a1-\uffff0-9]+-?)*[a-z\u??00a1-\uffff0-9]+)*(?:\.(?:[a-z\u00a1-\uffff]{2,})))(?::\d{2,5})?(?:\/[^\s]*)?$/i;

https://gist.github.com/jpillora/7885636

IMO, the longer, the better
but I still don't know how to retrieve the matched URL from it

[Chris]

This is one of many side-ripping comments in the Proboards source so I remembered it immediately upon reading this thread


// URL
else if (field.validation === 'url') {
validation = function(value) {
// There is a God, and he writes regular expressions o_o
return (value === '' || value.match(/^https?\:\/\/(([\w\!\~\*\'\(\)\.\$\%\&\+\=\-]+\:)?([\w\!\~\*\'\(\)\.\$\%\&\+\=\-]+\@))?(([0-9]{1,3}\.){3}[0-9]{1,3}|(([\w!~\*\'\(\)\=]+\.)*(\w[\w\-]{0,61})?\w\.[a-zA-Z]{2,6}))(\:[1-9][0-9]{0,4})?[\w\~\!\@\#\$\%\^\&\*\(\)\-\+\=\/\:\;\?\.\,]*$/i)) ? true : false;
}
validationMessage = 'be a valid URL';
}


obviously remove the start-of-line (^) and end-of-line ($) metacharacters and add a g flag to adapt to multi-search use.
Disclaimer: I haven't personally used this

[DarkPikachu]

haha alright
thanks

any idea what I'd have to do to get the values for the replacement text??

'<a href="$2" target=“_blank">$1$2</a>$3'

[Chris]

Nov 15, 2014 22:02:40 GMT -8 DarkPikachu said:

EDIT2:
also got this one:
var isValidUrl = /^(??:https?|ftp):\/\/)(?:\S+(?::\S*)?@)?(??!10(?:\.\d{1,3}){3})(?!127(?:\.??\d{1,3}){3})(?!169\.254(?:\.\d{1,3}){2})(?!192\.168(?:\.\d{1,3}){2})(?!172\.(?:1[??6-9]|2\d|3[0-1])(?:\.\d{1,3}){2})(?:[1-9]\d?|1\d\d|2[01]\d|22[0-3])(?:\.(?:1?\d{1??,2}|2[0-4]\d|25[0-5])){2}(?:\.(?:[1-9]\d?|1\d\d|2[0-4]\d|25[0-4]))|(??:[a-z\u00??a1-\uffff0-9]+-?)*[a-z\u00a1-\uffff0-9]+)(?:\.(?:[a-z\u00a1-\uffff0-9]+-?)*[a-z\u??00a1-\uffff0-9]+)*(?:\.(?:[a-z\u00a1-\uffff]{2,})))(?::\d{2,5})?(?:\/[^\s]*)?$/i;

https://gist.github.com/jpillora/7885636

IMO, the longer, the better
but I still don't know how to retrieve the matched URL from it

try running it against an url in a browser console ( as match() ) and check the returned value array for which captures to hone in on

[DarkPikachu]

tried your code reference on my browser console:

var isValidUrl = /^https?\:\/\/(([\w\!\~\*\'\(\)\.\$\%\&\+\=\-]+\:)?([\w\!\~\*\'\(\)\.\$\%\&\+\=\-]+\@))?(([0-9]{1,3}\.){3}[0-9]{1,3}|(([\w!~\*\'\(\)\=]+\.)*(\w[\w\-]{0,61})?\w\.[a-zA-Z]{2,6}))(\:[1-9][0-9]{0,4})?[\w\~\!\@\#\$\%\^\&\*\(\)\-\+\=\/\:\;\?\.\,]*$/i

same as above:

var url = "tcll5850.proboards.com";
undefined
url.match(isValidUrl)
null



var url = "http://tcll5850.proboards.com";
undefined
url.match(isValidUrl)
["http://tcll5850.proboards.com", undefined, undefined, undefined, "tcll5850.proboards.com", undefined, "tcll5850.proboards.com", "tcll5850.", "proboard", undefined]


my last code reference does the same thing >_>
havn't tried my first yet <_<

EDIT:
just did my first reference, here's the returns:
["tcll5850.proboards.com"]
["http://tcll5850.proboards.com"]

for what I know, it doesn't get port numbers though

[DarkPikachu]

oh and I forgot about one last important aspect... heh

how do I check that the URL is not already a linked URL??
(such as what's already been parsed from the [ a] and [ url] BBC tags)

[Chris]

I can tell you right now that trying to match an url with no protocol won't work since they are not optional in either of those regex expressions (only the s in http"s" is optional). The opportunities for false positives in situations without a protocol would be too great and would probably require a full fledged token parser but from the array it looks like the Proboards one breaks it down into capture parts for you. The other one actually commits to no captures that I can see since it starts everything with the anonymous capture "?:" pattern

[Chris]

Nov 15, 2014 22:52:34 GMT -8 DarkPikachu said:

oh and I forgot about one last important aspect... heh

how do I check that the URL is not already a linked URL??
(such as what's already been parsed from the [ a] and [ url] BBC tags)

That would depend on which environment you are working with. If you have a DOM tree then it should be simple to identify any anchors, but if you're working in text then you would need a separate regex that first harvests that (as you intimated when you said "my plugin converts [ a] and [ url] tags to <a ...> before the code I have checks for unlinked URLs" )

[DarkPikachu]

Nov 15, 2014 23:01:15 GMT -8 Chris said:

Nov 15, 2014 22:52:34 GMT -8 DarkPikachu said:

oh and I forgot about one last important aspect... heh

how do I check that the URL is not already a linked URL??
(such as what's already been parsed from the [ a] and [ url] BBC tags)

That would depend on which environment you are working with. If you have a DOM tree then it should be simple to identify any anchors, but if you're working in text then you would need a separate regex that first harvests that (as you intimated when you said "my plugin converts [ a] and [ url] tags to <a ...> before the code I have checks for unlinked URLs" )

and I realize I worded that wrong at that last part of the last sentence...

my code:
.replace(/\[a(.+?)\]/g,'<a$1>')
.replace(/\[\/a\]/g,'</a>')

.replace(/\[url\](.+?)\[\/url\]/g,'<a href="$1" target=“_blank">$1</a>')
.replace(/\[url=(.+?)\]/g,'<a href="$1" target=“_blank">')
.replace(/\[\/url\]/g,'</a>')

.replace(/( |\n)(http(|s):\/\/|www.)(.+?)( |\n)/g,'<a href="$2" target=“_blank">$1$2</a>$3')

Nov 15, 2014 22:56:29 GMT -8 Chris said:

I can tell you right now that trying to match an url with no protocol won't work since they are not optional in either of those regex expressions (only the s in http"s" is optional). The opportunities for false positives in situations without a protocol would be too great and would probably require a full fledged token parser but from the array it looks like the Proboards one breaks it down into capture parts for you. The other one actually commits to no captures that I can see since it starts everything with the anonymous capture "?:" pattern

then can you provide me with PB's regex please, because this is allowed:
tcll5850.proboards.com

^that link obviousely works here, but does not work in the code you provided me.

I'm going for EXACT PB behavior in my plugin :/

EDIT:
that includes what DOESN'T work with links on PB

with the code that works:
regex101.com

^this link does not work here, but the working code returns:
["regex101.com"]

meaning users will type that in thinking it works when it really doesn't.

yes I'm talking about my real-time preview plugin... >3<
I'm thinking about removing the preview tab for people who have this plugin active.
(it's been discussed many times with my forum's users and, currently reviewed, about 63% of them hate the preview tab)

[DarkPikachu]

so... anything??

you guys know I can't get the src for how PB handles links...

and no return notification while left hanging on the hook has me on delay...

do you want my plugin to display posts like PB displays them??
or do you want users to experience incompatibilities with my plugin??

I may be the developer, but the ones to blame here are the srcs for development...
(that meaning who/what I'm developing for is to blame for not instructing me properly)


I don't see what's so hard with just handing me the (proper) code so I can do stuff right.
(what Eaton posted earlier I was mislead to believe was how PB validates it's links)
^ obviousely that was proven false when tcll5850.proboards.com didn't work in that code.

so... what... we just stop there??
forcing me to use a code I found that seems to validate more links than PB validates??
is that how you guys what my plugin to behave??
causing many noobs to complain to me about things not working like PB with my plugin...

you realize I'll have to direct them here to show them the poor help I'm getting right??
I don't think that'd look good on PB's rep...
I'm trying to avoid that, but if this keeps up, I'll have no choice but to go through with that. :/

logic... people... :/

[Boy_Wonder]

Then maybe the idea here is RATHER then pushing the boat out and constantly demanding PB 'fixes' things that aren't broken to allow you to break rules you AGREE to in their Terms of Service and the rules for developing plugins learn to code within their constraints. Everyone of your posts I've seen is demanding people to help you code things or demanding people change things to suit you. If you're that desperate to get so advanced maybe the idea is proboards might not even be perfect for you. So rather then threaten and blackmail and generally throw your toys out the pram learn patience and learn to work WITHIN the rules set out. And actually bother learning and experimenting.

[DarkPikachu]

Nov 17, 2014 10:50:12 GMT -8 Boy_Wonder said:

demanding PB 'fixes' things that aren't broken

ok for one... I've stopped that whole trying to break rules stuff a while back ok.
and sorry for that.

but for the point of that quote, I'd say PB's visual previewer is pretty broken, so much so that it's better to just remove it altogether... heh
but that's just me, I know other people don't post PB's BBC on other forums.

other forums don't support [ font color] or [ a], they only support [ color] or [ url].
PB's previewer breaks my BBC in that the same code I intend to post on other forums [ color] and [ url] gets changed to PB's code.
that's one reason I'm building this plugin.

another is because RTEs generally break or duplicate BBC no matter how you use them.


but as for patience, ok, maybe I lost my cool a little too soon... I mean, my last post was only last night, and this IS a forum environment...
sorry for that...
but do keep in mind with stuff like that, I'm autistic, meaning I don't keep my mind-frame but for only a tad before switching to another major project.


and one last thing, I don't hate PB... they don't make nearly as poor decisions as MS.
but there are poor decisions made by PB which shouldn't simply be ignored.
>>> do stuff right and make things easy, and I won't have anything to complain about.


I'm not trying to threaten either... I'm simply stating what is...
I'll have to direct those noobs here to show them why my plugin does that...
that's just logic... it's not a threat... :/

[Boy_Wonder]

Nov 17, 2014 11:20:29 GMT -8 DarkPikachu said:

Nov 17, 2014 10:50:12 GMT -8 Boy_Wonder said:

demanding PB 'fixes' things that aren't broken

ok for one... I've stopped that whole trying to break rules stuff a while back ok.
and sorry for that.

but for the point of that quote, I'd say PB's visual previewer is pretty broken, so much so that it's better to just remove it altogether... heh
but that's just me, I know other people don't post PB's BBC on other forums.

other forums don't support [ font color] or [ a], they only support [ color] or [ url].
PB's previewer breaks my BBC in that the same code I intend to post on other forums [ color] and [ url] gets changed to PB's code.
that's one reason I'm building this plugin.

another is because RTEs generally break or duplicate BBC no matter how you use them.


but as for patience, ok, maybe I lost my cool a little too soon... I mean, my last post was only last night, and this IS a forum environment...
sorry for that...
but do keep in mind with stuff like that, I'm autistic, meaning I don't keep my mind-frame but for only a tad before switching to another major project.


and one last thing, I don't hate PB... they don't make nearly as poor decisions as MS.
but there are poor decisions made by PB which shouldn't simply be ignored.
>>> do stuff right and make things easy, and I won't have anything to complain about.


I'm not trying to threaten either... I'm simply stating what is...
I'll have to direct those noobs here to show them why my plugin does that...
that's just logic... it's not a threat... :/

I'm on autistic spectrum to. I have Aspergers Syndrome. And no its threatening. Because you can't get your own way.

It's 5 minutes TOPS to change code to the format for other forums. Note SHIELD has 2-3 versions of advertising code so I can just copy-paste the appropriate one. No qualms for me.

[DarkPikachu]

._.
ok well I'm NOT trying to threaten...
but I have to give those noobs SOME fallback...

anyways... why change the code in the first place...
I'm on about 37 forums (1/3 of those being PB-based) and no other non-PB forum I'm on gives me any hastle with BBC when I have to preview it.
(because no other non-PB forum I'm on accepts [ font] or [ a], PB borks my BBC, and it shouldn't)

plus, why should I have to undo what PB does... I use the formal BBC when I write my posts [ color] [ url]... and I expect it to remain when I submit the post.
I have no problems here when I don't use the visual editor, my BBC is exactly how I want it, and stays in support for other forums as well)

the visual editor is broken, and most of my members don't even use it. (thanks to my real-time BBC previewer)
but to me, it's not good enough to release until it can display posts EXACTLY like PB displays them.