Differences

This shows you the differences between two versions of the page.

Link to this comparison view

irc:1436306400 [2017/05/27 13:44] (current)
Line 1: Line 1:
 +[05:46:02] *** ChanServ sets mode: +o purplefox_
 +
 +[10:26:12] *** ChanServ sets mode: +o cescoffier
 +
 +[10:26:30] *** ChanServ sets mode: +o pmlopes
 +
 +[10:26:47] *** ChanServ sets mode: +o temporal_
 +
 +[17:39:18] <​purplefox_>​ pmlopes: cescoffier; Hi folks. I am trying to build the website locally, but it's failing, complaining about some npm dependency not existing:
 +
 +[17:39:35] <​cescoffier>​ do you have the dependency name ?
 +
 +[17:39:36] <​purplefox_>​ any ideas?
 +
 +[17:39:36] <​purplefox_>​ [INFO] Running 'npm install --color=false'​ in /​home/​tim/​projects/​vert-x3/​vertx-web-site
 +
 +[17:39:37] <​purplefox_>​ [ERROR] npm ERR! Linux 3.13.0-37-generic
 +
 +[17:39:37] <​purplefox_>​ [ERROR] npm ERR! argv "/​home/​tim/​projects/​vert-x3/​vertx-web-site/​node/​node"​ "/​home/​tim/​projects/​vert-x3/​vertx-web-site/​node/​npm/​bin/​npm-cli.js"​ "​install"​ "​--color=false"​
 +
 +[17:39:37] <​purplefox_>​ [ERROR] npm ERR! node v0.12.2
 +
 +[17:39:37] <​purplefox_>​ [ERROR] npm ERR! npm  v2.7.6
 +
 +[17:39:39] <​purplefox_>​ [ERROR] npm ERR! code ETARGET
 +
 +[17:39:41] <​purplefox_>​ [ERROR]
 +
 +[17:39:43] <​purplefox_>​ [ERROR] npm ERR! notarget No compatible version found: metalsmith-collections at '>​=0 dot 0.6 <​0.0.7'​
 +
 +[17:40:01] <​purplefox_>​ I did a "mvn clean site" to trigger this
 +
 +[17:40:05] <​cescoffier>​ let me try, you may have an outdated node / npm
 +
 +[17:40:51] <​cescoffier>​ (cannot be an outdated node/npm we install them)
 +
 +[17:43:07] <​cescoffier>​ hum, works on my machine. What's weird in the metalsmith-collections version, because it's 0.7
 +
 +[17:43:19] <​cescoffier>​ did you try to do a mvn clean install first ?
 +
 +[17:43:31] <​cescoffier>​ nope
 +
 +[17:43:36] <​cescoffier>​ found !
 +
 +[17:43:52] <​cescoffier>​ can you try one small edit ?
 +
 +[17:43:59] <​cescoffier>​ in the package.json remove line 32
 +
 +[17:45:10] <​aesteve>​ npm...
 +
 +[17:45:36] <​aesteve>​ btw "​request":​ "​^2.55.0",​
 +
 +[17:46:03] <​aesteve>​ In the past I got a TON of issues due to this "​^"​
 +
 +[17:46:09] <​aesteve>​ I recommend not using it
 +
 +[17:46:28] <​cescoffier>​ aesteve: do you mean updating it or just get rid of it ?
 +
 +[17:46:39] <​aesteve>​ fix the version number
 +
 +[17:46:42] <​aesteve>​ get rid of it
 +
 +[17:47:06] <​cescoffier>​ aesteve: I've no idea what depends on this, need to ask Michael
 +
 +[17:47:12] <​purplefox_>​ cescoffier: you want me to remove this line: "​metalsmith-collections":​ "​^0.0.6",​ ?
 +
 +[17:47:20] <​aesteve>​ get rid of the "​^",​ not the dependency ;)
 +
 +[17:47:20] <​cescoffier>​ purplefox_: yes
 +
 +[17:47:29] <​cescoffier>​ aesteve: oh ok :-)
 +
 +[17:47:52] <​aesteve>​ I don't understand why it's the default behavior it doesn'​t make sense to me
 +
 +[17:48:40] <​purplefox_>​ cescoffier: ok that seems to work now :)
 +
 +[17:48:43] <​aesteve>​ if your software is working with one version why in hell would you tell your dependency manager to "​let'​s fetch the latest version if there'​s one so my build is broken and I have absolutely no idea why"
 +
 +[17:48:58] <​purplefox_>​ aesteve: lol
 +
 +[17:49:01] <​cescoffier>​ purplefox_: ok gonna ccommit this (with aeste improvement too)
 +
 +[17:49:24] <​purplefox_>​ cescoffier: i wonder why it worked for you though and not for me
 +
 +[17:49:32] <​purplefox_>​ (previously)
 +
 +[17:49:37] <​cescoffier>​ aesteve: this is the "​broken"​ application of the semver
 +
 +[17:49:56] <​cescoffier>​ purplefox_: because the dependency is duplicated
 +
 +[17:50:05] <​cescoffier>​ so the question is why it is working for me and for jenkins
 +
 +[17:50:22] <​aesteve>​ npm misteries, shhhh, don't ask
 +
 +[17:50:22] <​purplefox_>​ maybe you already have an old version installed, which no longer exists?
 +
 +[17:50:49] <​cescoffier>​ purplefox_: I doubt, as I never touch to metalsmith before doing the blog thiny
 +
 +[17:51:00] <​aesteve>​ cescoffier: I didn't understand, sry
 +
 +[17:53:03] <​aesteve>​ actually this default behavior would work in a world where every lib is retro-compatible
 +
 +[17:53:27] <​aesteve>​ and javascript ecosystem is far far away from this world
 +
 +[17:54:47] <​cescoffier>​ aesteve: yes this is the semver
 +
 +[17:55:31] <​aesteve>​ ah ok, I read "​server"​
 +
 +[17:55:45] <​cescoffier>​ purplefox_: I had the '​go'​ from Mark to publish his post.
 +
 +[18:01:58] <​purplefox_>​ cescoffier: where is the PR?
 +
 +[18:02:04] <​purplefox_>​ can't see it
 +
 +[18:02:16] <​cescoffier>​ was the mangled PR
 +
 +[18:02:31] <​cescoffier>​ https://​github.com/​bytor99999/​vertx-web-site/​pull/​1
 +
 +[18:03:33] <​purplefox_>​ ok cool. would be good to see the unmangled pr though
 +
 +[18:07:02] <​cescoffier>​ purplefox_: anyway, I had to edit a bit the post
 +
 +[18:07:31] <​cescoffier>​ https://​github.com/​vert-x3/​vertx-web-site/​blob/​web-site-3.0.0/​src/​site/​blog/​posts/​vertx3-migration-from-vertx-2.1.x-part-one.md
 +
 +[18:07:42] <​cescoffier>​ (and also improved the BLOG.md guide)
 +
 +[18:08:29] <​cescoffier>​ purplefox: waiting your go to make it public
 +
 +[18:08:41] <​cescoffier>​ purplefox_: waiting your go to make it public
 +
 +[18:34:57] <​cristianmiranda>​ Hi guys, I have a question about worker verticle threads. I have an app which does heavy processing in one of the verticles (thre worker one) but in the logs I see that there is ONLY ONE thread for that worker. How can I specify more threads for that verticle? Thanks in advance and keep up the good work :)
 +
 +[18:36:03] <​purplefox_>​ cescoffier: looks fine to me
 +
 +[18:36:52] <​purplefox_>​ cristianmiranda:​ that's not the vert.x way. you don't more threads in a verticle as that could lead to race conditions, and a bit part of Vert.x is it protects you from race conditions.
 +
 +[18:36:58] <​purplefox_>​ what you want is to deploy more instances of the verticle
 +
 +[18:41:07] <​aesteve>​ purplefox_: this question comes very often lately
 +
 +[18:41:34] <​aesteve>​ in the google group someone asked the same question kindof
 +
 +[18:42:00] <​purplefox_>​ aesteve: i think this is to be expected as most java devs are used to multi-threaded concurrency approach and not usually so familiar with other concurrency e.g. actor model
 +
 +[18:42:18] <​purplefox_>​ and vert.x is unusual (for java world) in that it pushes a more actor-like model for concurrency
 +
 +[18:42:29] <​aesteve>​ yes probably, I struggled to explain that to one of my friends, too
 +
 +[18:43:05] <​aesteve>​ purplefox_: i have a question regarding benchmarking
 +
 +[18:43:34] <​aesteve>​ I'd like to benchmark my framework since it's invoking methods by reflection, there must be some overhead
 +
 +[18:44:04] <​purplefox_>​ i strongly recommend using jmh for this
 +
 +[18:44:36] <​purplefox_>​ it makes it easy to do comparative benchmarks without having to worry too much that the benchmark methodology is right
 +
 +[18:44:52] <​purplefox_>​ because most hand-rolled benchmarks i see are flawed
 +
 +[18:45:00] <​aesteve>​ ok thanks a lot I'll give it a try
 +
 +[18:45:05] <​purplefox_>​ http://​openjdk.java.net/​projects/​code-tools/​jmh/​
 +
 +[18:45:10] <​purplefox_>​ it's very easy to use
 +
 +[18:45:25] <​purplefox_>​ you just use a maven archetype to create your benchmarking project
 +
 +[18:45:30] <​aesteve>​ yes I was using apache benchmark and honestly i was doing it wrong, I had the exact same results
 +
 +[18:45:52] <​aesteve>​ (which doesn'​t make sense)
 +
 +[18:45:53] <​cristianmiranda>​ Sorry, not sure I understand. How can I get more instances of the verticle deployed?
 +
 +[18:45:55] <​purplefox_>​ ab sucks and i believe this is for http benchmarking
 +
 +[18:46:04] <​cristianmiranda>​ I only did deployVerticle just once
 +
 +[18:46:19] <​purplefox_>​ cristianmiranda:​ just deploy more instances
 +
 +[18:46:31] <​purplefox_>​ you can specify more instances when deploying in the options
 +
 +[18:46:35] <​purplefox_>​ or at the command line
 +
 +[18:46:38] <​aesteve>​ mmh actually I'm not sure how I will test it without http
 +
 +[18:46:40] <​cristianmiranda>​ Every time I need to do the calculations or just the times I need it at the beginning?
 +
 +[18:46:45] <​purplefox_>​ or just call deployVerticle multiple times
 +
 +[18:46:46] <​aesteve>​ but nvm I'll find a way, thanks
 +
 +[18:47:09] <​purplefox_>​ aesteve: well if you want to test just the overhead of reflection then why is http important?
 +
 +[18:48:03] <​aesteve>​ it's not :) it's just that I wanted to test the reflection in my framework (like for example, injection context parameters directly into the user's method)
 +
 +[18:48:04] <​cristianmiranda>​ Oh! the .setInstances(int)
 +
 +[18:48:16] <​cristianmiranda>​ Thank you guys! I'll give it a try
 +
 +[18:48:24] <​aesteve>​ and I'm not sure how I will do that
 +
 +[18:48:49] <​purplefox_>​ aesteve: i'd just creating a method that does something similar (uses reflection to inject params into an object)
 +
 +[18:48:57] <​purplefox_>​ and test that. don't see why you need http
 +
 +[18:49:08] <​purplefox_>​ and then compare that to setting the params directly
 +
 +[18:49:36] <​purplefox_>​ most likely reflection will be negligible overhead compared to IO which is why you can't see the difference using ab
 +
 +[18:50:11] <​aesteve>​ yes, I'll do that, in fact I need some "​mock"​ routingContext that emulates what I'm doing in my framework
 +
 +[18:50:27] <​aesteve>​ it's not really reflection, but class/​instanceof testing
 +
 +[18:51:03] <​aesteve>​ example : when I encounter "​Vertx"​as method parameter, I replace it with routingContext.vertx()
 +
 +[18:51:18] <​aesteve>​ so there'​s a lof of "​parameter.getClass().equals(...)"​
 +
 +[18:51:34] <​aesteve>​ that's the stuff I need to test
 +
 +[18:52:07] <​aesteve>​ but thanks, I'll do that using jmh, and thats a new tool to play with :)
 +
 +[18:57:49] <​aesteve>​ another example : https://​github.com/​ldallen/​Nubes-UseCases/​blob/​master/​simpleREST/​src/​main/​java/​treeptik/​ldallen/​controller/​todolist/​ProductsController.java#​L49-L53
 +
 +[18:58:12] <​aesteve>​ that's this kind of stuff I'd like to make sure is efficient
 +
 +[18:59:41] <​cristianmiranda>​ Guys, I've tried the .setInstances.. the thing is that the app is doing the same thing that amount of times.. I just need more threads available or whatever exists to calculate different things but at the same time.
 +
 +[19:01:09] <​aesteve>​ "​calculate different things at the same time" is a little vague :\
 +
 +[19:08:03] <​cristianmiranda>​ aesteve: Let me explain a bit more, sorry :). The app listents to requests to do calculations over an entity ID which is later fetched from mongoDB. There will be a lot of requests per second and at the moment I only see one thread dedicated for the calculations (which are encapsulared in a verticle).
 +
 +[19:08:25] <​cristianmiranda>​ What I would need now is a way of doing those calculations in parallell, let's say for ID 2, 3 and 4 at the same time
 +
 +[19:09:54] <​aesteve>​ What if your request server is a standard verticle (non worker) and your "​calculator"​ a worker verticle with a many instances
 +
 +[19:10:15] <​aesteve>​ then from the server, when a request arrives, you send a message to the worker verticle to do the calculation
 +
 +[19:10:40] <​aesteve>​ (a message through the event bus)
 +
 +[19:11:05] <​cristianmiranda>​ Should I use send instead of publish?
 +
 +[19:11:16] <​cristianmiranda>​ That's why I see the calculations for the same entity happening 20 times?
 +
 +[19:12:30] <​purplefox_>​ yes, send means "sent to one instance"​. publish means "send to all instances"​ (assuming they listen at the same address)
 +
 +[19:12:51] <​purplefox_>​ alternatively you can use executeblocking and don't need worker verticles at all
 +
 +[19:13:51] <​cristianmiranda>​ Perfect. I'll test the send for now and see how it performs. Thank you!
 +
 +[19:14:04] <​purplefox_>​ np
 +
 +[19:14:11] <​aesteve>​ same as usual with Verticle vs NotAVerticle stuff. The good question is : does your "​calculator"​ exist on its own, is this a standalone service
 +
 +[19:14:48] <​aesteve>​ if yes, it should probably be a Verticle, if not, don't bother accessing it through the eventbus and use executeBlocking
 +
 +[19:15:36] <​aesteve>​ purplefox_: do you agree with this definition ? I think I might write a blog post someday
 +
 +[19:19:39] <​cristianmiranda>​ aesteve: Yeah, it's a standalone module that can live on its own, it just needs an input to calculate and then it saves the result to mongoDB
 +
 +[19:20:16] <​aesteve>​ but are you going to ask him to do calculations from another application ? another node on the network ?
 +
 +[19:22:08] <​aesteve>​ because if you aren'​t,​ it could be a standard "​jar"​ dependency you use as "new Calculator(vertx)"​
 +
 +[19:25:11] <​cristianmiranda>​ There are two apps (one is a vertx app and one is not).. the vertx app gets the requests from the non vertx app and do the calculations. The idea is to dispatch the request when it comes and handle them in parallell.
 +
 +[19:26:40] <​aesteve>​ I think the vertx app doesn'​t need a worker verticle
 +
 +[19:27:29] <​aesteve>​ when you receive a request, simply call vertx.executeBlocking and when the result is calculated your handler will be called
 +
 +[19:29:39] <​cristianmiranda>​ aesteve: But I like the idea of controlling the amount of instances of the worker verticle. Is there a huge performance difference between the two approaches?
 +
 +[19:30:11] <​aesteve>​ you xan do the same with the instances of the non-worker verticle :)
 +
 +[19:30:25] <​aesteve>​ you SHOULD, actually :)
 +
 +[19:31:08] <​aesteve>​ if you want to handle a lot of concurrency just use as many eventloops as available
 +
 +[19:31:41] <​cristianmiranda>​ So, I won't deploy the verticle as a worker, but I need to wrap the "​calculation step in the verticle"​ into a vertx.executeBlocking thing? That's what you're saying?
 +
 +[19:32:25] <​aesteve>​ that's what purplefox_ suggested, I won't steal from him, I'm way to afraid of doing it
 +
 +[19:32:58] <​cristianmiranda>​ aesteve: haha, cool. I'll try that then. And will use the eventloop instead of the worker.
 +
 +[20:20:41] <​purplefox_>​ cristianmiranda:​ please don't forget to read the docs about executeBlocking before implementing anything
 +
 +[20:22:07] <​cristianmiranda>​ I will :) Thanks!
 +
 +[22:29:08] *** ChanServ sets mode: +o temporalfox