I have created 2 queues in Rabbit. First if main, second should be used when Exception is thrown in consumer. On older Symfony with ampq bundle is all ok, but in 6.2 with Messenger not. There is problem with JsonSerializer. I "copy/paste" solution recomended by manual of Symfony.
Lets have first queue named queue with this parameters (second is queue_retry, but it doesnt matter at this time):
arguments:
x-dead-letter-exchange: 'queue_retry'
x-dead-letter-routing-key: ''
And class for JsonSerializer
class RabbitMessageMessengerJsonSerializer implements SerializerInterface
{
public function decode(array $encodedEnvelope): Envelope
{
$body = $encodedEnvelope['body'];
$headers = $encodedEnvelope['headers'];
$data = json_decode($body, true);
$message = new RabbitMessageMessenger($data);
$envelope = new Envelope($message);
$stamps = [];
if (isset($headers['stamps'])) {
$stamps = unserialize($headers['stamps']);
}
return $envelope->with(... $stamps);
}
public function encode(Envelope $envelope): array
{
$message = $envelope->getMessage();
if ($message instanceof RabbitMessageMessenger) {
$data = $message->getContent();
} else {
throw new \Exception('Unsupported message class');
}
$allStamps = [];
foreach ($envelope->all() as $stamps) {
$allStamps = array_merge($allStamps, $stamps);
}
return [
'body' => json_encode($data),
'headers' => [
'stamps' => serialize($allStamps),
],
];
}
}
When I run consumer and throw Exception, I got this message:
In RabbitMessageMessengerJsonSerializer.php line 46:
[Exception] Serialization of 'Closure' is not allowed
In this case is message "copied" and exists in both queues. I need to take it from "queue" and when it fails "move" to "queue_retry". And without problems.
If I remove stamps (set to empty array) message is copied into "queue" many times until I brak running of consumer.
What is wrong?
Thank you very much
Related
I've been using Symfony 3.1 with new Cache Component (https://symfony.com/doc/current/components/cache.html), I'm using the redis adapter
config.yml
cache:
app: cache.adapter.redis
default_redis_provider: "redis://127.0.0.1:6379"
Basically, I save data in redis when I do a GET to a specific resource, and I remove it from redis, when I do a POST..
With symfony in dev mode data is stored/removed from cache as I expected.
But when I change it to prod, the 'deleteItem' no longer removes the item from redis cache..
I cannot find any error in logs, so I'm getting a little bit lost with it..
This is a sample of how I'm using the cache
protected function getCache(){
return $this->get('cache.app');
}
public function getAction(){
$cacheItem = $this->getCache()->getItem('example-key');
$data = ... // Check cacheItem isHit() ...
$cacheItem->expiresAfter($this->defaultCacheTime);
$cacheItem->set($data);
$this->getCache()->save($cacheItem);
}
public function postAction() {
...
$this->getCache()->deleteItem('example-key');
}
Update - I've found what might be causing this issue
This is part of the code of symfony AbstractAdapter and RedisAdapter:
public function deleteItem($key)
{
return $this->deleteItems(array($key));
}
public function deleteItems(array $keys)
{
$ids = array();
foreach ($keys as $key) {
$ids[$key] = $this->getId($key);
unset($this->deferred[$key]);
}
try {
if ($this->doDelete($ids)) {
return true;
}
} catch (\Exception $e) {
}
$ok = true;
// When bulk-delete failed, retry each item individually
foreach ($ids as $key => $id) {
try {
$e = null;
if ($this->doDelete(array($id))) {
continue;
}
} catch (\Exception $e) {
}
CacheItem::log($this->logger, 'Failed to delete key "{key}"', array('key' => $key, 'exception' => $e));
$ok = false;
}
return $ok;
}
protected function doDelete(array $ids)
{
if ($ids) {
$this->redis->del($ids);
}
return true;
}
This is part of code from Predis StreamConnection.php:
public function writeRequest(CommandInterface $command)
{
$commandID = $command->getId();
$arguments = $command->getArguments();
$cmdlen = strlen($commandID);
$reqlen = count($arguments) + 1;
$buffer = "*{$reqlen}\r\n\${$cmdlen}\r\n{$commandID}\r\n";
for ($i = 0, $reqlen--; $i < $reqlen; $i++) {
$argument = $arguments[$i];
$arglen = strlen($argument);
$buffer .= "\${$arglen}\r\n{$argument}\r\n";
}
$this->write($buffer);
}
When I call deleteItem('example-key'), it then calls the deleteItems(..) to remove that key..
The thing is, deleteItems() is calling doDelete() and passing an array like
'example-key' => 'prefix_example-key'
The doDelete(), then calls the Redis client, passing that same array string => string, when I think it should be, index => string,eg: [0] => 'prefix_example-key' instead of ['example-key'] => 'prefix_example-key'
Then the redis client when processing the command to execute, receives that array as $arguments, and in the for loop, it does this:
$argument = $arguments[$i]; since the array is in string => string format, it won't work, in dev mode, it shows Notice undefined offset 0 error
This is the strange part
In 'dev' mode, it will throw an error, and so, the deleteItems() will catch it, and try to delete the item again, this time, sending the arguments properly
In 'prod' mode, the Notice undefined offset 0 don't know why, but it does not throw an exception, so deleteItems(..) won't catch it, returns right there..
I've found a way to make it work for me, if I add array_values in the doDelete method, it works:
protected function doDelete(array $ids)
{
if ($ids) {
$this->redis->del(array_values($ids));
}
return true;
}
I don't know if all of this is making sense or not, I think I'll open an issue in symfony bug tracker
My bad, this was caused by an outdated version of Predis, I tought I had the latest version of Predis, but I didn't
Everything is working fine with the latest version
My controller code:
public function postFilesAction(Request $request)
{
$validator = $this->get('validator');
$requestCredentials = RequestCredentials::fromRequest($request);
$errors = $validator->validate($requestCredentials);
...
validate method in RequestCredentials (Callback constraint).
/**
* #Assert\Callback(payload = {"errorCode" = "FILE_FILE_URL"})
*/
public function validate(ExecutionContextInterface $context)
{
if (! ($this->fileExistsAndValid() || $this->fileUrlExistsAndValid())) {
$context->buildViolation('Neither file nor file_url is present.')->addViolation();
}
}
Callback works as expected, but the value of $constraintViolation->$constraint->$payload is null.
When I'm trying to use payload in other Constraints (NotBlank, for example), it works (I can see it in ConstraintViolation object).
Is it Symfony bug or am I doing somethings wrong? Should I use some other solution to my problem? (I need to check if there's at least one of two fields (file or file_url) present in request).
In Symfony 3.0 you cannot easily access the payload in the callback when using the Callback constraint. Starting with Symfony 3.1, the payload will be passed as an additional argument to the callback (see https://github.com/symfony/symfony/issues/15092 and https://github.com/symfony/symfony/pull/16909).
I managed to solve this problem with following code in the assertion:
/**
* #Assert\Callback(payload = {"error_code" = "1"}, callback = "validate", groups = {"Default", "RequestCredentials"})
*/
public function validate(ExecutionContextInterface $context)
{
// some validation code
}
I think the problem was because of the Symfony Callback constraint constructor:
public function __construct($options = null)
{
// Invocation through annotations with an array parameter only
if (is_array($options) && 1 === count($options) && isset($options['value'])) {
$options = $options['value'];
}
if (is_array($options) && !isset($options['callback']) && !isset($options['groups'])) {
$options = array('callback' => $options);
}
parent::__construct($options);
}
When it is given $options = ['payload' => [...]] (what happened in my case) it turns it into $options = ['callback' => ['payload' => [...]]]
and then '$payload' data becomes inacessable in ConstraintViolation object.
But I'm still not sure whether it's Symfony imperfection or me not getting something and using it wrong.
My simple data transformer transforms a number to an entity and vice-versa. It's pretty like the example in the official documentation.
The reverseTransform method converts a number to an entity and when it fails it throws a TransformationFailedException with a descriptive message:
public function reverseTransform($number)
{
if (!$number) {
return null;
}
$issue = $this->om
->getRepository('AcmeTaskBundle:Issue')
->findOneBy(array('number' => $number))
;
if (null === $issue) {
throw new TransformationFailedException(sprintf(
'An issue with number "%s" does not exist!',
$number
));
}
return $issue;
}
However the form field using the above transformer gets a generic error message "This value is not valid". Even changing the exception text (that I would expect to be used as validation message, but it's not...) doesn't change the error message.
How can I display the exception text instead of "This value is not valid"?
In no way, because symfony catch this exception and set own message (field incorrect).
If your want customize this message, you must set validator to this field.
Maybe I'm wrong, but did not find anything.
For example:
public function reverseTransform($number)
{
if (!$number) {
return null;
}
$issue = $this->om
->getRepository('AcmeTaskBundle:Issue')
->findOneBy(array('number' => $number))
;
if (null === $issue) {
// Nothig action
//throw new TransformationFailedException(sprintf(
// 'An issue with number "%s" does not exist!',
// $number
//));
}
return $issue;
}
And add NotBlank/NotNull validator to field.
UPD
And you can set the parameter "invalid_message" in form type.
For example:
$builder
->add('you_field', 'text', array('invalid_message' => 'An issue number not found'))
->get('you_field')->addModelTransformer('....');
I build a model side validation in Laravel 4 with the creating Model Event :
class User extends Eloquent {
public function isValid()
{
return Validator::make($this->toArray(), array('name' => 'required'))->passes();
}
public static function boot()
{
parent::boot();
static::creating(function($user)
{
echo "Hello";
if (!$user->isValid()) return false;
});
}
}
It works well but I have issues with PHPUnit. The two following tests are exactly the same but juste the first one pass :
class UserTest extends TestCase {
public function testSaveUserWithoutName()
{
$count = User::all()->count();
$user = new User;
$saving = $user->save();
assertFalse($saving); // pass
assertEquals($count, User::all()->count()); // pass
}
public function testSaveUserWithoutNameBis()
{
$count = User::all()->count();
$user = new User;
$saving = $user->save();
assertFalse($saving); // fail
assertEquals($count, User::all()->count()); // fail, the user is created
}
}
If I try to create a user twice in the same test, it works, but it's like if the binding event is present only in the first test of my test class. The echo "Hello"; is printed only one time, during the first test execution.
I simplify the case for my question but you can see the problem : I can't test several validation rules in different unit tests. I try almost everything since hours but I'm near to jump out the windows now ! Any idea ?
The issue is well documented in Github. See comments above that explains it further.
I've modified one of the 'solutions' in Github to automatically reset all model events during the tests. Add the following to your TestCase.php file.
app/tests/TestCase.php
public function setUp()
{
parent::setUp();
$this->resetEvents();
}
private function resetEvents()
{
// Get all models in the Model directory
$pathToModels = '/app/models'; // <- Change this to your model directory
$files = File::files($pathToModels);
// Remove the directory name and the .php from the filename
$files = str_replace($pathToModels.'/', '', $files);
$files = str_replace('.php', '', $files);
// Remove "BaseModel" as we dont want to boot that moodel
if(($key = array_search('BaseModel', $files)) !== false) {
unset($files[$key]);
}
// Reset each model event listeners.
foreach ($files as $model) {
// Flush any existing listeners.
call_user_func(array($model, 'flushEventListeners'));
// Reregister them.
call_user_func(array($model, 'boot'));
}
}
I have my models in subdirectories so I edited #TheShiftExchange code a bit
//Get all models in the Model directory
$pathToModels = '/path/to/app/models';
$files = File::allFiles($pathToModels);
foreach ($files as $file) {
$fileName = $file->getFileName();
if (!ends_with($fileName, 'Search.php') && !starts_with($fileName, 'Base')) {
$model = str_replace('.php', '', $fileName);
// Flush any existing listeners.
call_user_func(array($model, 'flushEventListeners'));
// Re-register them.
call_user_func(array($model, 'boot'));
}
}
I asked this question and found out that we can't get the error message thrown by a DataTransformer (according to the only user who answered, maybe it's possible, I don't know).
Anyway, now that I know that, I am stucked with a problem of validation. Suppose my model is this one: I have threads that contains several participants (users).
<?php
class Thread
{
/**
* #ORM\Column(name="id", type="integer")
* #ORM\Id
* #ORM\GeneratedValue(strategy="AUTO")
*/
private $id;
/**
* #ORM\ManyToMany(targetEntity="My\UserBundle\Entity\User")
* #ORM\JoinTable(name="messaging_thread_user")
*/
private $participants;
// other fields, getters, setters, etc
}
For thread creation, I want the user to specify the participants usernames in a textarea, separated by "\n".
And I want that if one or more of the usernames specified don't exist, a message is displayed with the usernames that don't exist.
For example, "Users titi, tata and toto don't exist".
For that I created a DataTransformer that transforms the raw text in the textarea into an ArrayCollection containing instances of users. Since I can't get the error message provided by this DataTransformer (such a shame! Is it really impossible?), I don't check the existence of each usernames in the DataTransformer but in the Validator.
Here is the DataTransformer that converts \n-separated user list into an ArrayCollection (so that the DataBinding is ok):
<?php
public function reverseTransform($val)
{
if (empty($val)) {
return null;
}
$return = new ArrayCollection();
// Extract usernames in an array from the raw text
$val = str_replace("\r\n", "\n", trim($val));
$usernames = explode("\n", $val);
array_map('trim', $usernames);
foreach ($usernames as $username) {
$user = new User();
$user->setUsername($username);
if (!$return->contains($user)) {
$return->add($user);
}
}
return $return;
}
And here is my validator:
<?php
public function isValid($value, Constraint $constraint)
{
$repo = $this->em->getRepository('MyUserBundle:User');
$notValidUsernames = array();
foreach ($value as $user) {
$username = $user->getUsername();
if (!($user = $repo->findOneByUsername($username))) {
$notValidUsernames[] = $username;
}
}
if (count($notValidUsernames) == 0) {
return true;
}
// At least one username is not ok here
// Create the list of usernames separated by commas
$list = '';
$i = 1;
foreach ($notValidUsernames as $username) {
if ($i < count($notValidUsernames)) {
$list .= $username;
if ($i < count($notValidUsernames) - 1) {
$list .= ', ';
}
}
$i++;
}
$this->setMessage(
$this->translator->transChoice(
'form.error.participant_not_found',
count($notValidUsernames),
array(
'%usernames%' => $list,
'%last_username%' => end($notValidUsernames)
)
)
);
return false;
}
This current implementation looks ugly. I can see the error message well, but the users in the ArrayCollection returned by the DataTransformer are not synchronized with Doctrine.
I got two questions:
Is there any way that my validator could modify the value given in parameter? So that I can replace the simple User instances in the ArrayCollection returned by the DataTransformer into instances retrieved from the database?
Is there a simple and elegant way to do what I'm doing?
I guess the most simple way to do this is to be able to get the error message given by the DataTransformer. In the cookbook, they throw this exception: throw new TransformationFailedException(sprintf('An issue with number %s does not exist!', $val));, if I could put the list of non-existing usernames in the error message, it would be cool.
Thanks!
I am the one that answered your previous thread so maybe someone else will jump in here.
Your code can be simplified considerably. You are only dealing with user names. No need for use objects or array collections.
public function reverseTransform($val)
{
if (empty($val)) { return null; }
// Extract usernames in an array from the raw text
// $val = str_replace("\r\n", "\n", trim($val));
$usernames = explode("\n", $val);
array_map('trim', $usernames);
// No real need to check for dups here
return $usernames;
}
The validator:
public function isValid($userNames, Constraint $constraint)
{
$repo = $this->em->getRepository('SkepinUserBundle:User');
$notValidUsernames = array();
foreach ($userNames as $userName)
{
if (!($user = $repo->findOneByUsername($username)))
{
$notValidUsernames[$userName] = $userName; // Takes care of dups
}
}
if (count($notValidUsernames) == 0) {
return true;
}
// At least one username is not ok here
$invalidNames = implode(' ,',$notValidUsernames);
$this->setMessage(
$this->translator->transChoice(
'form.error.participant_not_found',
count($notValidUsernames),
array(
'%usernames%' => $invalidNames,
'%last_username%' => end($notValidUsernames)
)
)
);
return false;
}
=========================================================================
So at this point
We have used transformer to copy the data from the text area and generated an array of user names during form->bind().
We then used a validator to confirm that each user name actually exists in the database. If there are any that don't then we generate an error message and form->isValid() will fail.
So now we are back in the controller, we know we have a list of valid user names (possibly comma delimited or possibly just an array). Now we want to add these to our thread object.
One way would to create a thread manager service and add this functionality to it. So in the controller we might have:
$threadManager = $this->get('thread.manager');
$threadManager->addUsersToThread($thread,$users);
For the thread manager we would inject our entity manager. In the add users method we would get a reference to each of the users, verify that the thread does not already have a link to this user, call $thread->addUser() and then flush.
The fact that we have wrapped up this sort of functionality into a service class will make things easier to test as we can also make a command object and run this from the command line. it also gives us a nice spot to add additional thread related functionality. We might even consider injecting this manager into the user name validator and moving some of the isValid code to the manager.