This version (2017/05/27 13:44) is a draft.
Approvals: 0/1

[00:12:54] <AlexLehm> temporal_: hi julien

[12:02:55] * ChanServ sets mode: +o temporalfox [12:52:39] <AlexLehm> temporalfox: the https proxy pr is finished i would say, there is one small issue that is related to a netty function that is not correct I would think [12:52:53] <AlexLehm> for the normal use it works however [12:53:12] <AlexLehm> I wonder if I should add a note to the documentation about that [12:57:56] <temporalfox> AlexLehm can you explain ? you mean netty feature ? [12:58:07] <temporalfox> if PR is ready feel free to submit [12:58:33] <AlexLehm> i have submitted the pr yesterday evening https://github.com/eclipse/vert.x/pull/1405 [12:59:04] <AlexLehm> the proxy function currently has an issue when you try to access a server that doesn't resolve in the local dns or resolves to another ip address then on the internet [13:00:56] <AlexLehm> e.g. assume you are running your tool in an intranet with an address 10.x and uses a proxy that has an address on that network and also an address in the internet. if the dns for a domain, e.g. www.yourcompany.com resolves to a 10.x address in the intranet, the proxy will try to access the ip address as CONNECT 10.x.x.x:443, which may not work since the public address of the web page would be resolved on the proxy itself [13:01:40] <AlexLehm> I changed the proxy code in vertx to use an unresolved InetSocketAddress, but netty resolves the address in the connect code so that you get the wrong address or a host unknown error [13:02:37] <AlexLehm> i will write a bug report for netty about that, for now this will not work just like it doesn't work in netty directly [13:12:21] <AlexLehm> if the address resolves to a ipv6 address but the proxy only supports ipv4, that would be the same isue i think [17:46:30] * ChanServ sets mode: +o temporalfox

[21:14:36] <temporalfox> AlexLehm hi

[21:16:00] <AlexLehm> hello Julien

[21:16:47] <temporalfox> sorry I lost track of what you told me earlier today

[21:17:01] <temporalfox> can you discuss this again now ?

[21:17:07] <temporalfox> about the DNS

[21:18:07] <AlexLehm> can i get back to you in bout 20 minutes? i can write a summary of the issue

[21:18:14] <temporalfox> sure

[21:18:27] <temporalfox> ping me again on #vertx

[21:18:32] <temporalfox> so I get your attention

[21:18:38] <temporalfox> well rather

[21:18:40] <temporalfox> you get mine :-)

[21:46:38] <AlexLehm> temporalfox: ok, I'm back

[21:46:45] <temporalfox> hi again

[21:48:19] <AlexLehm> ok, the issue is related to the fact that a proxy may resolve hostnames differently that the client accessing the proxy. currently netty resolves the hostname to the address in the client before sending the request to the proxy, that works for most applications, but in some cases it is wrong

[21:48:44] <temporalfox> a couple of questions

[21:48:49] <temporalfox> how is this resolution in netty ? blocking ?

[21:48:59] <temporalfox> can you point me out the netty code doing that ?

[21:49:04] <AlexLehm> the resolution is netty is non-blocking

[21:49:11] <temporalfox> where is it ?

[21:49:16] <temporalfox> class / line

[21:51:29] <AlexLehm> i have a test class for that somewhere, let me check

[21:53:09] <temporalfox> ok

[21:55:16] <AlexLehm> this is Bootstrap.doResolveAndConnect https://github.com/netty/netty/blob/4.1/transport/src/main/java/io/netty/bootstrap/Bootstrap.java#L159

[21:56:06] <AlexLehm> the address is resolved in line 174

[21:56:10] <temporalfox> ok

[21:56:30] <temporalfox> so why did we need to have something similar in vertx if the bootstrap does it ?

[21:57:15] <AlexLehm> the problem is bootstrap shouldn't do it in the proxy case

[21:57:42] <AlexLehm> the vertx code creates an unresolved address, which is correct and netty notices that and tries to resolve it

[21:58:32] <AlexLehm> it works in most cases and the resolution is correct, but not in the intranet config

[21:59:08] <temporalfox> you mean that on an intranet a proxy could resolve something that the client could not ?

[21:59:15] <temporalfox> or it would resolve it with a different IP ?

[21:59:15] <AlexLehm> yes

[21:59:40] <AlexLehm> both may happen, that depends on the dns config of the intranet

[21:59:45] <AlexLehm> worst case there may be none

[22:00:10] <temporalfox> can you give me an usual case ?

[22:01:53] <temporalfox> actually we do use it on

[22:01:56] <temporalfox> server binding

[22:02:07] <temporalfox> however we do configure the resolver in Vert.x

[22:02:25] <temporalfox> in case of proxy I think we could maybe set a null resolver on the Bootstrap ?

[22:03:27] <temporalfox> note that

[22:03:32] <temporalfox> in Bootstrap

[22:03:48] <temporalfox> ah I'm trying to find which resolver is used by default

[22:03:54] <temporalfox> I'm seing this

[22:03:54] <temporalfox> private static final AddressResolverGroup<?> DEFAULT_RESOLVER = DefaultAddressResolverGroup.INSTANCE;

[22:04:01] <temporalfox> and

[22:04:01] <temporalfox> private volatile AddressResolverGroup<SocketAddress> resolver =

[22:04:03] <temporalfox> (AddressResolverGroup<SocketAddress>) DEFAULT_RESOLVER;

[22:05:41] <AlexLehm> with a different resolver, the address will also resolve to something wrong or what happens when it doesn't resolve?

[22:06:31] <temporalfox> actually in my case

[22:06:40] <temporalfox> when I wrote the similar code that is in netty proxy http

[22:06:48] <temporalfox> is that the connect was done with the proxy address

[22:07:00] <temporalfox> and then my handler was using the unresolved value

[22:07:22] <temporalfox> because netty proxy http takes the connect address

[22:07:30] <temporalfox> and transforms it into a request to the proxy

[22:07:37] <temporalfox> so it uses the connect pipieline

[22:08:10] <temporalfox> a workaround I think

[22:08:18] <temporalfox> is to add an handler before the http proxy handler

[22:08:30] <temporalfox> and in this hander transforms the connect address to the proxy handler

[22:08:35] <temporalfox> or

[22:08:40] <temporalfox> subclass the proxy http

[22:08:51] <temporalfox> to use the unresolve address

[22:09:18] <AlexLehm> would it be possible to provide a resolver that doesn't actually resolve the address?

[22:09:30] <temporalfox> I do'nt know

[22:09:37] <AlexLehm> the proxy http supports using the unresolved address, i think

[22:10:46] <AlexLehm> something like vertx proxy code creates unresolved address → Bootstrap resolver creates a new unresolved address → proxy handler evaluates hostname from unresolved address

[22:12:27] <temporalfox> I'm looking at something like this

[22:12:43] <temporalfox> if we could override

[22:12:44] <temporalfox> public final void connect(

[22:12:45] <temporalfox> ChannelHandlerContext ctx, SocketAddress remoteAddress, SocketAddress localAddress,

[22:12:45] <temporalfox> ChannelPromise promise) throws Exception {

[22:12:51] <temporalfox> we could change the remoteAddress

[22:13:21] <temporalfox> maybe we can override

[22:13:21] <temporalfox> destinationAddress

[22:13:44] <temporalfox> public final <T extends SocketAddress> T destinationAddress()

[22:13:46] <temporalfox> final everywhere

[22:14:41] <temporalfox> what we could override is

[22:14:49] <temporalfox> nothing

[22:14:56] <temporalfox> it's a damn final class

[22:15:07] <temporalfox> only option would be to add an handler before the proxy

[22:15:10] <temporalfox> and change the connect address

[22:15:49] <temporalfox> i.e transform the resolve address into an unresolve address

[22:17:56] <temporalfox> can you provide a case for this ?

[22:18:03] <temporalfox> a test case

[22:18:15] <temporalfox> like you connect with a name

[22:18:27] <temporalfox> and in the proxy you assert that the name is the name and not an address

[22:18:33] <temporalfox> like using “localhost”

[22:18:44] <temporalfox> and check that's it's not resolved to 127.0.0.1

[22:18:56] <temporalfox> (I think the initial impl will fail)

[22:19:49] <AlexLehm> https://gist.github.com/alexlehm/9a339fe0223c0adb1ed12ceb0a12dbd2

[22:20:36] <AlexLehm> this currenty gets an unknownhostexception in the bootstrap code, but it should fail in the proxy class

[22:21:19] <temporalfox> why don't you use “localhost” ?

[22:22:16] <AlexLehm> i wanted to access an unknown host, i like to add a status in the proxy to return the exception

[22:22:25] <temporalfox> ok

[22:22:31] <AlexLehm> we can use localhost as well

[22:22:55] <temporalfox> can you make such case in your branch and then one of us try to fix it ?

[22:23:50] <AlexLehm> ah, the test is not starting an actual test server, so localhost will not work

[22:23:55] <AlexLehm> i can change that

[22:23:56] <temporalfox> then we can fix it I see how

[22:24:05] <AlexLehm> ok

[22:24:20] <temporalfox> no need to open a netty issue

[22:24:21] <temporalfox> it exists

[22:24:28] <temporalfox> but it is just not documented :-)

[22:36:14] <temporalfox> when you have the test

[22:36:19] <temporalfox> let me know

[22:36:27] <temporalfox> I mean it should not take much long

[22:36:36] <temporalfox> and we can certainly finish that tonight if you can

[22:37:53] <AlexLehm> ok

[22:40:40] <AlexLehm> ok, i commit the changed test

[22:41:53] <AlexLehm> maybe it would work to create a subclass of the SocketAddress class

[22:43:04] <truk77_> HTTP server question: If I have a more than one route that matches a given path, can I have the first route to handle the request add headers to the request before passing it to the next handler?

[22:43:26] <temporalfox> actually

[22:43:30] <temporalfox> you can try to fix it yourself

[22:43:33] <temporalfox> with this change

[22:44:00] <temporalfox> in ProxyChannelProvider

[22:44:22] <temporalfox> add

[22:44:23] <temporalfox> bootstrap.resolver(NoopAddressResolverGroup.INSTANCE)

[22:44:28] <temporalfox> in connect(

[22:44:46] <AlexLehm> ah, thats what i meant by a resolver that doesn't resolve I think

[22:44:52] <temporalfox> yes

[22:45:16] <temporalfox> the doc of NoopAddressResolver says

[22:45:19] <temporalfox> * A {@link AddressResolver} that does not perform any resolution but always reports successful resolution.

[22:45:19] <temporalfox> * This resolver is useful when name resolution is performed by a handler in a pipeline, such as a proxy handler.

[22:45:37] <temporalfox> so that's why I said to not open a netty issue

[22:45:44] <temporalfox> as they expect the case to happen

[22:45:51] <temporalfox> and one should set this resolver on the bootstrap

[22:45:58] <AlexLehm> ok

[22:46:07] <temporalfox> I think actually we should set it in all vertx bootstrap

[22:46:11] <temporalfox> as we do the resolution before

[22:46:23] <temporalfox> (so it would avoid to create them for nothing)

[22:46:40] <temporalfox> truk77_ I do'nt think you can modify the HttpServerRequest

[22:46:45] <AlexLehm> if it is already resolved, the code does not do a resolve I think

[22:46:51] <temporalfox> yes

[22:46:56] <temporalfox> it just passes the address

[22:47:07] <temporalfox> and trust that the SocketChannel will connect

[22:56:22] <AlexLehm> ok, i have commited the fix, the jenkins is currently hangin

[22:56:30] <temporalfox> which jenkins ?

[22:57:03] <temporalfox> you have your own jenkins ?

[22:57:15] <AlexLehm> no, the one from vertx

[22:57:20] <AlexLehm> from cloudbees

[22:57:35] <temporalfox> which job is hanging ?

[22:57:39] <temporalfox> ah

[22:57:40] <AlexLehm> all

[22:57:48] <temporalfox> did you set it up ?

[22:58:07] <temporalfox> what does trigger this build ?

[22:58:21] <AlexLehm> i am starting that manually since it is my fork repo

[22:58:28] <temporalfox> ok

[22:58:36] <temporalfox> what do you use for this ?

[22:58:54] <temporalfox> I've been observing problems with clustering test

[22:58:57] <temporalfox> that fails

[22:59:13] <AlexLehm> i commit to the alexlehm/vert.x and then run the whole test on the jenkins

[22:59:18] <temporalfox> does the fix I suggested make the test pass ?

[22:59:28] <temporalfox> I didn't know you had access to cloudbees CI

[22:59:29] <AlexLehm> yes it does when i run it locally

[22:59:35] <temporalfox> ok cool

[22:59:52] <AlexLehm> Tim created a user for me for that when i started the mail project

[23:00:03] <temporalfox> ah yes

[23:00:09] <temporalfox> makes sense

[23:00:12] <temporalfox> how do you run just a build for yourself ?

[23:00:22] <temporalfox> which part of the user interface does that ?

[23:01:05] <AlexLehm> i created a job with my github address and I can rn that with the build button

[23:01:18] <AlexLehm> or you can click on the project and the on build now

[23:03:35] <AlexLehm> actually its a normal build, its just a different project

[23:04:46] <temporalfox> ok

[23:04:50] <temporalfox> but I don't see it in this list

[23:05:00] <temporalfox> https://vertx.ci.cloudbees.com/view/vert.x-3/

[23:08:34] <AlexLehm> i think its in the all view only https://vertx.ci.cloudbees.com/job/alexlehm-vert.x3-core-issue1361/

[23:20:31] <temporalfox> ok

[23:20:51] <temporalfox> I will go to bed soon AlexLehm can you comment on the PR this change ?

[23:21:14] <AlexLehm> you mean commit?

[23:21:40] <AlexLehm> I did that already

[23:21:53] <temporalfox> ok commit is fine

[23:22:03] <temporalfox> I've done a couple of commentso n the PR

[23:22:09] <temporalfox> about doc

[23:22:38] <AlexLehm> yes, i am changing that now

[23:22:48] <temporalfox> should be good for merge soon

[23:24:13] <AlexLehm> very good

[23:24:35] <AlexLehm> I have a day off tomorrow so I may be able to do some stuff if necessary

[23:33:16] <AlexLehm> ok, commit the stuff for now