Suppose I have pretrained the network on other database. Because of overfitting problem, my database is not very diversity , so the overfitting problem is very serious. I want to load the pretrained network parameter in chainer v2.0 but to fix first several layers, in this case, what should I use in chainer v2.0, I know in chainer1.0 there is volatile keyword but which is deprecated in v2.0.
Should I use with chainer.no_backprop_mode(): inside def __call__ when process in first several layers?
Yes, you can use chainer.no_backprop_mode() context manager in your forward computation code to fix parameters of specific layers. Here is an example:
def __call__(self, x):
with chainer.no_backprop_mode():
h1 = F.relu(self.l1(x))
h2 = F.relu(self.l2(h1))
return self.l3(h2)
Related
I am using a CNN architecture and adding few more layers in between and creating a new model. By using model.summary() on this new model, I see everything is aligned fine but in tensorboard, I am seeing duplicate blocks connected around with new blocks.Kind of new network parallely drawn with the old network block and some connection in between.
I am using tensorboard = Tensorboard() for the tensorboard in keras.
Please advise why I see these connections and old network blocks in parallel with new model blocks but on the other hand model.summary() looks totally fine.
I am trying to understand so any detail on this will help.
I have faced the similar issue.
The main reason for this is, whenever the model is created, every layer gets a new name. For example: if you have a model with two 2d-convolutional layers followed by a dense layer. Creating the model for the 1st time and executing model.Summary(), results in the below layer names:
conv2d_1
conv2d_2
dense_1
While re-executing the same code, yields:
conv2d_3
conv2d_4
dense_2
At the same time, the logs that Tensor-board uses are overwritten. Hence, parallel blocks of layers with different names appears.
I'm trying to implement my own version of the Asynchronous Advantage Actor-Critic method, but it fails to learn the Pong game. My code was mostly inspired by Arthur Juliani's and OpenAI Gym's A3C versions. The method works well for a simple Doom environment (the one used in Arthur Juliani's code), but when I try the Pong game, the method diverges to a policy where it always executes the same action (always move down, or always move up, or always executes the no-op action). My code is located in my GitHub repository.
I have already adapted my network to resemble the architecture used by OpenAI Gym's A3C version, which is:
4 convolutional layers with the same specs, those being: 32 filters, 3x3 kernels, 2x2 strides, with padding (padding='same'). The output of the last convolutional layer is then flattened and fed to a LSTM layer with an output of size 256. The initial states C and H of the LSTM layer are given as an input. The output of the LSTM layer is then separated into two streams: a fully connected layer with an output size equals to the number of actions (policy) and another fully connected layer with only one output (value function) (more details in Network.py of my code);
The loss function used is just as is informed in the original A3C paper. Basically, the policy loss is the log_softmax of the linear policy times the advantage function. The value loss is the square of the difference between the value function and the discounted rewards. The total loss accounts for the value loss, policy loss, and the entropy. The gradients are clipped to 40 (more details in Network.py of my code);
There is only one global network and several worker networks (one network for each worker). Only the global network is updated. This update is done with respect to the local gradients of each worker network. Therefore, each worker simulate the environment for BATCH_SIZE iterations, saving the state, value function, chosen action, reward received, and the LSTM state. After BATCH_SIZE (I used BATCH_SIZE = 20) iterations, each worker pass those data into the network, calculate the discounted rewards, the advantage function, the total loss, and the local gradients. It then updates the global network with those gradients. Finally, the worker's local network is synchronized with the global network (local_net = global_net). All workers does that asynchronously (for more details in this step, check the work and train methods of the Worker class inside the Worker.py);
The LSTM states C and H are reset between episodes. It is also important to note that the current states C and H are kept locally by each worker;
To apply the gradients to the global network, I used the Adamoptimizer with learning rate = 1e-4.
I have already tried different configurations for the network (by trying several different convolutional layers configurations, including different activation functions), other optimizers (RMSPropOptimizer and AdadeltaOptimizer) with different parameters configurations, and different values to BATCH_SIZE. But it almost ends up diverging to a policy where it always executes only one action. I mean always because there are certain configurations where the agent maintains a policy similar to a random policy for several episodes, with no apparent improvements (I waited until 62k episodes before giving up in those cases).
Therefore, I would like to know if anyone have obtained success in training an agent in the Pong game using the A3C with a LSTM layer. If so, what are the parameters used? Any help would be appreciated!
[EDIT] As I said in the comments, I managed to partially solve the problem by feeding the correct LSTM state before calculating the gradients (instead of feeding an initialized LSTM state). This made the method learn reasonably well for the PongDeterministic environment. But the problem persists when I try the Breakout-v0: the agent reaches a mean score of 40 in about 65k episodes, but it seems to stop learning after this (it maintained this score for some time). I have checked the OpenAI starter agent several times and I can't find any significant differences between mine implementation with their's. Any help would be extremely appreciated!
How can defined topology in Castalia-3.2 for WBAN ?
How can import topology in omnet++ to casalia ?
where the topology defined in default WBAN scenario in Castalia?
with regard
thanks
Topology of a network is an abstraction that shows the structure of the communication links in the network. It's an abstraction because the notion of a link is an abstraction itself. There are no "real" links in a wireless network. The communication is happening in a broadcast medium and there are many parameters that dictate if a packet is received or not, such as the power of transmission, the path loss between transmitter and receiver, noise and interference, and also just luck. Still, the notion of a link could be useful in some circumstances, and some simulators are using it to define simulation scenarios. You might be used to simulators that you can draw nodes and then simply draw lines between them to define their links. This is not how Castalia models a network.
Castalia does not model links between the nodes, it models the channel and radios to get a more realistic communication behaviour.
Topology is often confused with deployment (I confuse them myself sometimes). Deployment is just the placement of nodes on the field. There are multiple ways to define deployment in Castalia, if you wish, but it is not needed in all scenarios (more on this later). People can confuse deployment with topology, because under very simplistic assumptions certain deployments lead to certain topologies. Castalia does not make these assumptions. Study the manual (especially chapter 4) to get a better understanding of Castalia's modeling.
After you have understood the modeling in Castalia, and you still want a specific/custom topology for some reason then you could play with some parameters to achieve your topology at least in a statistical sense. Assuming all nodes use the same radios and the same transmission power, then the path loss between nodes becomes a defining factor of the "quality" of the link between the nodes. In Castalia, you can define the path losses for each and every pair of nodes, using a pathloss map file.
SN.wirelessChannel.pathLossMapFile = "../Parameters/WirelessChannel/BANmodels/pathLossMap.txt"
This tells Castalia to use the specific path losses found in the file instead of computing path losses based on a wireless channel model. The deployment does not matter in this case. At least it does not matter for communication purposes (it might matter for other aspects of the simulation, for example if we are sampling a physical process that depends on location).
In our own simulations with BAN, we have defined a pathloss map based on experimental data, because other available models are not very accurate for BAN. For example the, lognormal shadowing model, which is Castalia's default, is not a good fit for BAN simulations. We did not want to enforce a specific topology, we just wanted a realistic channel model, and defining a pathloss map based on experimental data was the best way.
I have the impression though that when you say topology, you are not only referring to which nodes could communicate with which nodes, but which nodes do communicate with which nodes. This is also a matter of the layers above the radio (MAC and routing). For example it's the MAC and Routing that allow for relay nodes or not.
Note that in Castalia's current implementations of 802.15.6MAC and 802.15.4MAC, relay nodes are not allowed. So you can not create a mesh topology with these default implementations. Only a star topology is supported. If you want something more you'll have to implemented yourself.
I am in need of synchronizing models/forwarding models between different computers. The models can represent tables but also trees (entries with child items). The setup will be:
Server providing a TCP Service (or other communication)
keep a list of models that need to be synchronized (registerSourceModel() in the like of proxy models)
provide the modellist with unique IDs to clients
provides QDataStream for packet serialization
Client(s) providing a TCP Connection to the server
create NetworkModel instances based on modellist from server
forward / provide any data() queries to the model
forward / provide any setData() queries to the model
... rest of typical model methods
There are two options of setting up the synchronization:
Have an QAbstractItemModel (may QStandardItemModel) with a complete cache (duplication of all data) on the client side, just updating on remote dataChanged()/layoutChanged() signals
Directly forward any requests over the network resulting in stub (Loading***) data entries until the real data has been fetched (dataChanged() signals) from the network
While option 1) will have quite the memory print, since we will have to duplicate any relevant model, it will most of the time have a very short response time, since synchronization can be done in background when needed.
Option 2) will have little to (almost) no memory usage, since everything will be queried directly. I am still not sure how this will actually look&feel when having big models to be visualized in a view. Think about an company catalogue (or list of amazon articles with some details) being required to be queried one by one (data() works on top of a single QModelIndex) over network.
As a result we probably will go with the first option.
The problems I encounter in both options are the synchronization/forwarding of a valid QModelIndex. These are always invalid on the remote computer. I did some research on the QSortFilterProxyModel() as this is kind of similar but within the same process space. This model keeps a identifying list for all indicies for mapping the "filtered" indicies.
Will this require myself to keep a identifying list of QPersistantModelIndex() on the server and a mapped list of IDs to my own QModelIndex() (with synchronization about these unique IDs of the list)?
Is there another option to "link" two models or even just put the whole model into some container and pipe that into my stream?
So we can use isObservable to work with some systems but I wonder If we can make tham show staps or do something like that. Is that possible?
I'm supposing that you are actually referring to the Observable and Controllable commands which are part of the DynamicSystems package.
Some Maple routines have so-called userinfo messages, which are optionally displayed and which can show partial steps. Those would typically be enabled with a call like infolevel[DynamicSystems]:=6 where 6 is the highest level of detail. With a lower value less detail might be displayed. Unfortunately, these particular DynamicSystems routines don't seem to have any userinfo messages in them. So this won't help directly in your case.
Sometimes one package calls another. The routine LinearAlgebra:-Rank is called, at some point, when Controllable is called. I'm guessing that you don't wish to see userinfo messages in Rank, which would be enabled by issuing infolevel[LinearAlgebra]:=n for some n between 1 and 6.
An alternative is to set printlevel high. But doing so will make all Maple internal calls also be verbose. The resulting output of setting, say, printlevel:=20 is overwhelming. I doubt that you'd find this approach useful.
Another alternative is to set certain routines as option trace. Sometimes doing so entails knowing the names of the appropriate internal routines. And this is made more complicated by the fact that not all non-exported module locals are visible by default. You could try first issuing the call, trace(DynamicSystems::ControllableSS): before invoking Controllable on your system. That shows some intermediary results, but those might not make much sense to you unless you know what source lines are generating those results. You can see the source by issuing the command, showstat(DynamicSystems::ControllableSS);
Here's an example,
restart:
with( DynamicSystems ):
aSys := StateSpace( <<1,2>|<3,4>>, <<2,3>>, <<1,0>|<0,1>>, <<0,0>> ):
aSys:-a, aSys:-b, aSys:-c:
trace(DynamicSystems::ControllableSS):
Controllable( aSys );
showstat(DynamicSystems::ControllableSS);
By looking at the source of the ControllableSS routine, you can deduce that it calls DynamicSystems:-StaircaseTransformation when the staircase method is specified. So here too you could issue showstat(DynamicSystems::StaircaseTransformation) to see the source of that internal (local) routine. Or you could trace that routine as well.
As a general rule, you can refer to exports of a module or package using the :- notation. And you can pass such names to trace and showstat using the :: notation instead of :-, unless you have first set kernelopts(opaquemodules=false). If an inner submodule member has more than one :- in its name then you'd have to use round-bracket delimiters to refer to it with the :: syntax. Sorry, that that's not so easy to express.